Linear RegressionΒΆ

Linear Regression algorithm aims to find parameters \(p_0\) and \(p_1\) for a line, \(y = p_0 + p_1 \cdot t\), that best fits \(N\) data points. The task is equivalent to solve systems of linear equations

\[\begin{split}Ap = \begin{bmatrix} 1&t_1 \\ 1&t_2 \\ \vdots&\vdots \\ 1&t_N \end{bmatrix} \begin{bmatrix} p_0 \\ p_1 \end{bmatrix} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_N \end{bmatrix} = Y.\end{split}\]

The method of least squares is the most common method for finding the fitted parameters. If \(A\) is of full column rank, the least squares solution is

\[p = (A^T A)^{-1} A^T Y\]

Input Parameters

Parameter Type Constraint Description Remarks
\([t_i]\) \([t_i] \in \mathbb R^N\) \(N \in \mathbb{N}\) None None
\(Y\) \(Y \in \mathbb R^N\) \(N \in \mathbb{N}\) Input data vector of length \(N\) None

Output Parameters

Parameter Type Constraint Description Remarks
\(p\) \(p \in \mathbb R^2\) None None None
\(\hat{Y}\) \(\hat{Y} \in \mathbb R^N\) \(N \in \mathbb{N}\) Output data vector of length \(N\) None

Single Steps using the Algorithm

References

  • C.R. Rao, H. Toutenburg, A. Fieger, C. Heumann, T. Nittner and S. Scheid, Linear Models: Least Squares and Alternatives, Springer Series in Statistics, pp. 23-33, 1999.
  • R.C. Aster, B. Borchers, C.H. Thurber, Parameter Estimation and Inverse Problems, Academic Press, 2005.