<s>[INST] Least Squares solution for a symmetric singular matrix I want to solve this system by Least Squares method:$$\begin{pmatrix}1 & 2 & 3\\\ 2 & 3 & 4 \\\ 3 & 4 & 5 \end{pmatrix}\begin{pmatrix}x\\y\\z\end{pmatrix} =\begin{pmatrix}1\\5\\-2\end{pmatrix} $$ This symmetric matrix is singular with one  eigenvalue $\lambda1 = 0$, so $\ A^t\cdot A$ is also singular and for this reason I cannot use the normal equation: $\hat x = (A^t\cdot A)^{-1}\cdot A^t\cdot b  $.
 So I performed Gauss-Jordan to the extended matrix to come with $$\begin{pmatrix}1 & 2 & 3\\\ 0 & 1 & 2 \\\ 0 & 0 & 0 \end{pmatrix}\begin{pmatrix}x\\y\\z\end{pmatrix} =\begin{pmatrix}1\\3\\-1\end{pmatrix} $$
Finally I solved the $\ 2x2$ system: $$\begin{pmatrix}1 & 2\\\ 0 & 1\end{pmatrix}\begin{pmatrix}x\\y\end{pmatrix} =\begin{pmatrix}1\\3\end{pmatrix} $$ taking into account that the best $\ \hat b\ $ is $\begin{pmatrix}1\\3\\0\end{pmatrix}$
The solution is then $\ \hat x = \begin{pmatrix}-5\\3\\0\end{pmatrix}$
Is this approach correct ? 
EDIT
Based on the book 'Lianear Algebra and its applications' from David Lay, I also include the Least Squares method he proposes: $(A^tA)\hat x=A^t b  $
$$A^t b =\begin{pmatrix}5\\9\\13\end{pmatrix}, A^tA = \begin{pmatrix}14 & 20 & 26 \\
20 & 29 & 38 \\
26 & 38 & 50\end{pmatrix}$$ 
The reduced echelon from the augmented is: 
$$ \begin{pmatrix}14 & 20 & 26 & 5 \\
20 & 29 & 38 & 9 \\
26 & 38 & 50 & 13 \end{pmatrix} \sim \begin{pmatrix}1 & 0 & -1 & -\frac{35}{6} \\
0 & 1 & 2 & \frac{13}{3} \\
0 & 0 & 0 & 0
\end{pmatrix} \Rightarrow \hat x = \begin{pmatrix}-\frac{35}{6} \\ \frac{13}{3} \\ 0 \end{pmatrix}$$ for the independent variable case that $z=\alpha ,  \alpha=0 $ 
 [/INST] The RLS solution is given by
$$ \hat x = A^+ \, b$$
where $A^+$ is the pseudo inverse of $A$.
As $A$ is not full rank, it is not possible effectively to calculate it with the simple formula you mentioned.
However, it is still possible to calculate it with numerical solutions, for example based on the SVD.
By doing so, we get:
$$ \hat x = \begin{pmatrix}-41/12\\-1/2\\29/12\end{pmatrix} $$
For a value $A \, \hat x$ equal to:
$$ A \, \hat x = \begin{pmatrix}17/6\\4/3\\-1/6\end{pmatrix} $$
Your method does not seem to work.  I can only give my interpretation of what happens:
The issue is your choice of the "best" $\hat b$ value. The vector that you considered is not directly related to the real $b$ vector, but to something obtained after some manipulations on the rows of the linear system matrix. 
Difficult in this situation  to rely it to a selection of a "best" $\hat b$ vector
</s>