Linear Regression

Global Algorithm - One-Dimensional Algorithm

Linear Regression algorithm aims to find parameters $p_0$ and $p_1$ for a line, $y = p_0 + p_1 \cdot t$, that best fits $N$ data points. The task is equivalent to solve systems of linear equations

$$\begin{split}Ap = \begin{bmatrix} 1&t_1 \\ 1&t_2 \\ \vdots&\vdots \\ 1&t_N \end{bmatrix} \begin{bmatrix} p_0 \\ p_1 \end{bmatrix} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_N \end{bmatrix} = Y.\end{split}$$

The method of least squares is the most common method for finding the fitted parameters. If $A$ is of full column rank, the least squares solution is

$$p = (A^T A)^{-1} A^T Y$$

Input Parameters

Parameter	Type	Constraint	Description	Remarks
$[t_i]$	$[t_i] \in \mathbb R^N$	$N \in \mathbb{N}$
$Y$	$Y \in \mathbb R^N$	$N \in \mathbb{N}$	Input data vector of length $N$

Output Parameters

Parameter	Type	Constraint	Description	Remarks
$p$	$p \in \mathbb R^2$
$\hat{Y}$	$\hat{Y} \in \mathbb R^N$	$N \in \mathbb{N}$	Output data vector of length $N$

Tool Support

• Maple

  For details refer to the online documentation of the function ‘LinearFit’.

• Matlab

  For details refer to the online documentation of the function ‘polyfit’.

Single Steps using the Algorithm

• Data Reduction With Linear Regression
• Reconstructing Improper Values with Linear Regression

References

• C.R. Rao, H. Toutenburg, A. Fieger, C. Heumann, T. Nittner and S. Scheid, Linear Models: Least Squares and Alternatives, Springer Series in Statistics, pp. 23-33, 1999.
• R.C. Aster, B. Borchers, C.H. Thurber, Parameter Estimation and Inverse Problems, Academic Press, 2005.