<s>[INST] Least Squares solution on a plane in $\mathbb R^3$ Let $W$ be the subspace of $\mathbb R^3$ spanned by $(1,2,3)^T$ and $(1,1,1)^T$. Find the point in $W$ which lies closest to $(-4,1,2)^T$
I know that the least squares solution is $(A^T)A = (A^T)b$.
However I do not know how to set up my matrix of $A$.
The solution to the problem is : the point closest to $b$ which lies in $\mathrm{Ran}(A)$ is $Ax = (-10/3, -1/3, 8/3)$.
How is this solved?
 [/INST] Let
$$
\mathbf{u}=\left[\begin{array}{c}
1\\
2\\
3
\end{array}\right]
$$
and
$$
\mathbf{v}=\left[\begin{array}{c}
1\\
1\\
1
\end{array}\right].
$$
We are trying to find $c$ and $d$ such that $c\mathbf{u}+d\mathbf{v}$
is closest (2-norm) to
$$
\mathbf{w}=\left[\begin{array}{c}
-4\\
1\\
2
\end{array}\right].
$$
That is,
$$
\min_{c,d}\left\Vert \left[\begin{array}{cc}
\mathbf{u} & \mathbf{v}\end{array}\right]\left[\begin{array}{c}
c\\
d
\end{array}\right]-\left[\mathbf{w}\right]\right\Vert=\min_{c,d}\left\Vert \left[\begin{array}{cc}
1 & 1\\
2 & 1\\
3 & 1
\end{array}\right]\left[\begin{array}{c}
c\\
d
\end{array}\right]-\left[\begin{array}{c}
-4\\
1\\
2
\end{array}\right]\right\Vert.
$$
As it turns out, this has the analytical solution (normal equations)
$$
\left[\begin{array}{cc}
1 & 1\\
2 & 1\\
3 & 1
\end{array}\right]^{T}\left[\begin{array}{cc}
1 & 1\\
2 & 1\\
3 & 1
\end{array}\right]\left[\begin{array}{c}
c\\
d
\end{array}\right]=\left[\begin{array}{cc}
1 & 1\\
2 & 1\\
3 & 1
\end{array}\right]^{T}\left[\begin{array}{c}
-4\\
1\\
2
\end{array}\right].
$$
</s>