<s>[INST] Quadratic fit with least square : any simple analytical expression? We consider the least square problem in the case where we got only one independant variable $x_i$ and only one dependant variable $y_i$. The number of observations is $n$.
In the case of the linear fit, we want to estimate $y_i$ with a function $f(x_i,µ) = µ_0 + x_i * µ_1$ under the constraint of minimizing $\sum_i{(y_i-f(x_i,µ))²}$.
The solutions can be expressed in the simple form :


*

*$µ_1 = \frac{covariance(x_i,y_i)_{i=1..n}}{variance(x_i)_{i=1..n}}$

*$µ_0 = \frac{(\overline{y}  - \overline{x} * µ_1)}n$


In the case of the quadratic fit, we got instead $f(x_i,µ) = µ_0 + x_i * µ_1 + x_i²*µ_2$.
Is there a way to express $µ_0$, $µ_1$ and $µ_2$ in an simple form ?
 [/INST] Simplicity is in the eye of the beholder.
Linear Fit
Linear system
$$
%
\begin{align}
% 
  \mathbf{A} \mu & = y \\
%
\left[
\begin{array}{cc}
 1 & x_{1} \\
 \vdots & \vdots \\
 1 & x_{m} \\
\end{array}
\right]
%
\left[
\begin{array}{c}
 \mu_{0} \\
 \mu_{1} \\
\end{array}
\right]
&=
%
\left[
\begin{array}{c}
 y_{1} \\
 \vdots \\
 y_{m} \\
\end{array}
\right]
%
\end{align}
%
$$
Normal equations
$$
%
\begin{align}
% 
  \mathbf{A}^{*} \mathbf{A} \mu & = \mathbf{A}^{*} y \\
%
\left[
\begin{array}{cc}
 \mathbf{1} \cdot \mathbf{1} & \mathbf{1} \cdot x \\
 x \cdot \mathbf{1} & x \cdot x \\
\end{array}
\right]
%
\left[
\begin{array}{c}
 \mu_{0} \\
 \mu_{1} \\
\end{array}
\right]
&=
\left[
\begin{array}{cc}
 \mathbf{1} \cdot y \\
 x \cdot y \\
\end{array}
\right]
%
\end{align}
$$
Least squares solution
$$
%
\begin{align}
%
\left[
\begin{array}{c}
 \mu_{0} \\
 \mu_{1} \\
\end{array}
\right]_{LS}
%
&=
%
\left( \det \left( \mathbf{A}^{*} \mathbf{A} \right) \right)^{-1}
%
\left[
\begin{array}{rr}
 x \cdot x & -\mathbf{1} \cdot x \\
 -\mathbf{1} \cdot x & \mathbf{1} \cdot \mathbf{1} \\
\end{array}
\right]
%
\left[
\begin{array}{cc}
 \mathbf{1} \cdot y \\
 x \cdot y \\
\end{array}
\right] \\
%
&=
\left( \det \left( \mathbf{A}^{*} \mathbf{A} \right) \right)^{-1}
\left[
\begin{array}{cc}
%
 \left( x \cdot x \right) \left( \mathbf{1} \cdot y \right) -
 \left( \mathbf{1} \cdot x \right) \left( x \cdot y \right) \\[4pt]
%
 \left( \mathbf{1} \cdot \mathbf{1} \right) \left( x \cdot y \right) -
 \left( \mathbf{1} \cdot x \right) \left( \mathbf{1} \cdot y \right) 
%
\end{array}
\right] 
%
\end{align}
%
$$

Quadratic fit
Linear system
$$
%
\begin{align}
% 
  \mathbf{A} \mu & = y \\
%
\left[
\begin{array}{ccc}
 1 & x_{1} & x^{2}_{1} \\
 \vdots & \vdots & \vdots \\
 1 & x_{m} & x^{2}_{m} \\
\end{array}
\right]
%
\left[
\begin{array}{c}
 \mu_{0} \\
 \mu_{1} \\
 \mu_{2} \\
\end{array}
\right]
&=
%
\left[
\begin{array}{c}
 y_{1} \\
 \vdots \\
 y_{m} \\
\end{array}
\right]
%
\end{align}
%
$$
Normal equations
$$
%
\begin{align}
% 
  \mathbf{A}^{*} \mathbf{A} \mu & = \mathbf{A}^{*} y \\
%
\left[
\begin{array}{rrr}
 \mathbf{1} \cdot \mathbf{1} & \mathbf{1} \cdot x & \mathbf{1} \cdot x^{2} \\
 x \cdot \mathbf{1} & x \cdot x & x \cdot x^{2} \\
 x^{2} \cdot \mathbf{1} & x^{2} \cdot x & x^{2} \cdot x^{2} \\
\end{array}
\right]
%
\left[
\begin{array}{c}
 \mu_{0} \\
 \mu_{1} \\
 \mu_{2} \\
\end{array}
\right]
&=
\left[
\begin{array}{r}
 \mathbf{1} \cdot y \\
 x     \cdot y \\
 x^{2} \cdot y \\
\end{array}
\right]
%
\end{align}
$$
Least squares solution
$$
%
\begin{align}
%
\left[
\begin{array}{c}
 \mu_{0} \\
 \mu_{1} \\
 \mu_{2} \\
\end{array}
\right]_{LS}
%
&=
\left( \det \left( \mathbf{A}^{*} \mathbf{A} \right) \right)^{-1}
\left[
\begin{array}{crc}
%
  \Sigma x^2   \Sigma x^5 - \left( \Sigma x^3\right)^2 &
  \Sigma x^2   \Sigma x^3 - \Sigma x   \Sigma x^4  &  
  \Sigma x     \Sigma x^3 - \left( \Sigma x^2\right)^2 \\
%
  \Sigma x^2   \Sigma x^3 - \Sigma x   \Sigma x^4  &   
m \Sigma x^4 - \left( \Sigma x^2 \right)^2 &
  \Sigma x   \Sigma x^2 - m   \Sigma x^3  \\
%
    \Sigma x   \Sigma x^3 - \left(\Sigma x^2\right)^2 & 
    \Sigma x   \Sigma x^2 - m   \Sigma x^3  &  
m   \Sigma x^2 - \left( \Sigma x \right)^2 \\
%
\end{array}
\right] 
%
\left[
\begin{array}{cc}
 \Sigma y \\
 \Sigma xy \\
 \Sigma x^{2}y \\
\end{array}
\right] \\
%
\end{align}
%
$$
</s>