<s>[INST] Largest singular values Given the positive semi-definite, symmetric matrix $A = bb^T + \sigma^2I$ where b is a column vector is it possible to find the singular values and singular vectors of the matrix analytically? I know that it has real eigenvalues since it's symmetric and positive semidefinite but not sure about solving directly for those values and their corresponding vectors.
 [/INST] The singular values are the eigenvalues of $A.$ By definition, when there exists a nonzero vector $\mathbf x$ for which $A\mathbf{x}=\lambda \mathbf{x},$ $\lambda$ is an eigenvalue and $\mathbf{x}$ is a corresponding eigenvector.
Note, then, that
$$A\mathbf{b} = (\mathbf{b}\mathbf{b}^\prime + \sigma^2I)\mathbf{b} = \mathbf{b}(\mathbf{b}^\prime \mathbf{b}) + \sigma^2 \mathbf{b} = (|\mathbf{b}|^2+\sigma^2)\mathbf{b},$$
demonstrating that $\mathbf{b}$ is an eigenvector with eigenvalue $\lambda_1 = |\mathbf{b}|^2 + \sigma^2.$
Furthermore, whenever $\mathbf{x}$ is a vector orthogonal to $\mathbf{b}$ -- that is, when $\mathbf{b}^\prime \mathbf{x} = \pmatrix{0},$ we may similarly compute
$$A\mathbf{x} = (\mathbf{b}\mathbf{b}^\prime + \sigma^2I)\mathbf{x} = \mathbf{b}(\mathbf{b}^\prime \mathbf{x}) + \sigma^2 \mathbf{x} = (0+\sigma^2)\mathbf{x},$$
showing that all such vectors are eigenvectors with eigenvalue $\sigma^2.$
Provided these vectors are in a finite dimensional vector space of dimension $n$ (say), a straightforward induction establishes that the vectors $x$ for which $\mathbf{b}^\prime \mathbf{x}=0$ form a subspace $\mathbf{b}^\perp$ of dimension $n-1.$  Let $\mathbf{e}_2, \ldots, \mathbf{e}_n$ be an orthonormal basis for this subspace.  It extends to an orthonormal basis $\mathscr{E} = (\mathbf{\hat  b}, \mathbf{e}_2, \ldots, \mathbf{e}_n)$ of the whole space where $\mathbf{\hat b} = \mathbf{b}/|\mathbf{b}|$.  In terms of this basis the matrix of $A$ therefore is
$$\operatorname{Mat}(A, \mathscr{E}, \mathscr{E}) = \pmatrix{|\mathbf{b}|^2+\sigma^2 & 0 & 0 & \cdots & 0 \\
0 & \sigma^2 & 0  & \cdots & 0 \\
0 & 0 & \ddots & \vdots & \vdots \\
\vdots & \vdots & \cdots & \ddots & 0 \\
0 & 0 & \cdots & 0 & \sigma^2
}$$
Whether or not every step of this derivation was clear, you can verify the result by setting
$$Q = \left(\mathbf{b}; \mathbf{e}_2; \ldots; \mathbf{e}_n\right)$$
to be the matrix with the the given columns and computing
$$Q\,\operatorname{Mat}(A, \mathscr{E}, \mathscr{E})\,Q^\prime = \mathbf{b}^\prime + \sigma^2I = A.$$
This is explicitly a singular value decomposition of the form $U\Sigma V^\prime$ where $V=Q,$ $\Sigma= \operatorname{Mat}(A, \mathscr{E}, \mathscr{E}),$ and $U=Q^\prime.$
The Gram Schmidt process provides a general algorithm to find $\mathscr{E}$ (and therefore $Q$): its input is the series of vectors $\mathbf{\hat b}$, $(1,0,\ldots,0)^\prime,$ and so on through $(0,\ldots,0,1)^\prime.$  After $n-1$ steps it will produce an orthonormal basis including the starting vector $\mathbf b.$

As an example, let $\mathbf{b} = (3,4,0)^\prime.$ With $\sigma^2 = 1,$ compute
$$\mathbf{b}\mathbf{b}^\prime + \sigma^2 I = \pmatrix{10&12&0\\12&17&0\\0&0&1}$$
Here, $|\mathbf{b}|^2 = 3^2+4^2+0^2=5^2,$ so that $\mathbf{\hat b} = \mathbf{b}/5 = (3/5,4/5,0)^\prime.$  One way to extend this to an orthonormal basis is to pick $\mathbf{e}_2 = (-4/5,3/5,0)^\prime$ and $\mathbf{e}_3 = (0,0,1)^\prime.$  Thus
$$Q = \pmatrix{3/5&4/5&0\\-4/5&3/5&0\\0&0&1}$$
and we may confirm that
$$\begin{align}
Q\,\operatorname{Mat}(A, \mathscr{E}, \mathscr{E})\,Q^\prime &= \pmatrix{3/5&4/5&0\\-4/5&3/5&0\\0&0&1}\pmatrix{5^2+1^2&0&0\\0&1&0\\0&0&1}\pmatrix{3/5&-4/5&0\\4/5&3/5&0\\0&0&1}\\
&=\pmatrix{10&12&0\\12&17&0\\0&0&1}
\end{align}$$
as intended.
</s>