Lineare Regression

Methode der kleinsten Quadrate

${\displaystyle RSS=\sum _{i=1}^{n}d_{i}^{2}=\sum _{i=1}^{n}(y_{i}-(a+bx_{i}))^{2}\rightarrow min!}$

bezüglich a und b.

Nach Ausmultiplikation, Ableiten und Nullsetzen

${\displaystyle {\frac {\partial S}{\partial a}}=-2\sum _{i=1}^{n}y_{i}+2na+2b\sum _{i=1}^{n}x_{i}=0,}$

${\displaystyle {\frac {\partial S}{\partial b}}=-2\sum _{i=1}^{n}x_{i}y_{i}+2a\sum _{i=1}^{n}x_{i}+2b\sum _{i=1}^{n}x_{i}^{2}=0,}$

erhält man die gesuchten Regressionskoeffizienten als die Lösungen

${\displaystyle b={\frac {\sum _{i=1}^{n}x_{i}y_{i}-n{\bar {x}}{\bar {y}}}{\sum _{i=1}^{n}x_{i}^{2}-n{\bar {x}}^{2}}}\;}$

und

${\displaystyle a={\bar {y}}-b{\bar {x}}\;,}$

wobei ${\displaystyle {\bar {x}}={\frac {1}{n}}\sum _{i=1}^{n}x_{i}}$.

Mit dem Verschiebungssatz kann man ${\displaystyle b}$ auch so ermitteln:

${\displaystyle b={\frac {\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}})}{\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}}}}$

Schätzungen ŷ

${\displaystyle {\hat {y}}_{i}=a+bx_{i}}$

Residuen r_i :

${\displaystyle {\begin{matrix}&y_{i}&=&a+bx_{i}+d_{i}&=&{\hat {y}}_{i}+d_{i}&&\\\Rightarrow &d_{i}&=&y_{i}-{\hat {y}}_{i}&\\\end{matrix}}}$

Stichprobenvarianz der Residuen:

${\displaystyle s^{2}={\frac {1}{n-2}}\sum _{i}d_{i}^{2}}$

Bestimmtheitsmaß

${\displaystyle r^{2}={\frac {{\frac {1}{n}}\sum _{i=1}^{n}({\hat {y}}_{i}-{\bar {y}})^{2}}{{\frac {1}{n}}\sum _{i=1}^{n}(y_{i}-{\bar {y}})^{2}}}={\frac {(\sum _{i=1}^{n}(x_{i}-{\bar {x}})(y_{i}-{\bar {y}}))^{2}}{\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}\sum _{i=1}^{n}(y_{i}-{\bar {y}})^{2}}}\;,}$

mit dem Verschiebungssatz :

${\displaystyle r^{2}={\frac {(\sum _{i=1}^{n}x_{i}y_{i}-n\cdot {\bar {x}}\cdot {\bar {y}})^{2}}{(\sum _{i=1}^{n}x_{i}^{2}-n\cdot {\bar {x}}^{2})(\sum _{i=1}^{n}y_{i}^{2}-n\cdot {\bar {y}}^{2})}}.}$

${\displaystyle 0\leq r^{2}\leq 1}$

Varianz der Residuen

${\displaystyle s^{2}={\frac {1}{n-2}}(1-r^{2})\cdot \sum _{i=1}^{n}(y_{i}-{\bar {y}})^{2}}$

Variablentransformation

Funktion	u	v
${\displaystyle y=a+b\cdot x^{n}}$	${\displaystyle x^{n}}$	${\displaystyle v=a+b\cdot u}$
${\displaystyle y={\frac {a}{b+x}}}$	${\displaystyle x}$	${\displaystyle v={\frac {1}{y}}={\frac {b}{a}}+{\frac {1}{b}}\cdot u}$
${\displaystyle y=a\cdot x^{b}}$	${\displaystyle ln(x)}$	${\displaystyle v=ln(y)=ln(a)+b\cdot ln(x)}$
${\displaystyle y=a\cdot b^{x}}$	${\displaystyle x}$	${\displaystyle v=ln(y)=ln(a)+x\cdot ln(b)}$
${\displaystyle y=a\cdot e^{b\cdot x}}$	${\displaystyle x}$	${\displaystyle v=ln(y)=ln(a)+b\cdot x}$

Die Methode kann auf weitere Parameter erweitert werden.