<s>[INST] Find 3×3 orthogonal matrix satisfying the following conditions. Let $A$ be 3×3 orthogonal real matrix.
For $\mathbf{u}$ $= \begin{pmatrix} 1 \\ 1 \\ 1 \end{pmatrix}, \mathbf{v}$ $= \begin{pmatrix} 0 \\ 0 \\ \sqrt3 \end{pmatrix}, \mathbf{w}$ $= \begin{pmatrix} 1 \\ -1 \\ 0 \end{pmatrix}$,
$Au=v, Aw=-w$ and $detA=-1$
Find $A$.
I noted that $u, v, w$ are linearly independent. And I write
$A\begin{pmatrix}
  1 & 0 & 1 \\
  1 & 0 & -1 \\
  1 & \sqrt3 & 0
 \end{pmatrix}=\begin{pmatrix}
  0 & a & -1 \\
  0 & b & 1 \\
  \sqrt3 & c & 0
 \end{pmatrix}.$
Then $A=\begin{pmatrix}
  0 & a & -1 \\
  0 & b & 1 \\
  \sqrt3 & c & 0
 \end{pmatrix}\begin{pmatrix}
  \frac{1}{2} & \frac{1}{2} & 0 \\
  -\frac{1}{2\sqrt3} & -\frac{1}{2\sqrt3} & \frac{1}{\sqrt3} \\
  \frac{1}{2} & -\frac{1}{2} & 0
 \end{pmatrix}$
Now, how can I find $a, b, c$??
Am I doing this right?
If there is a better solution, please help me. 
 [/INST] Your approach is fine; you can proceed by using the fact that $\det(A)=-1$ and $A$ is orthogonal. From
$$-1=\det(A)=\det\begin{pmatrix}
  0 & a & -1 \\
  0 & b & 1 \\
  \sqrt3 & c & 0
 \end{pmatrix}\det\begin{pmatrix}
  \frac{1}{2} & \frac{1}{2} & 0 \\
  -\frac{1}{2\sqrt3} & -\frac{1}{2\sqrt3} & \frac{1}{\sqrt3} \\
  \frac{1}{2} & -\frac{1}{2} & 0
 \end{pmatrix},$$
you find that $a+b=-2$, so you can substitute $b=-(a+2)$. Now multiply the two matrices and choose $a$ and $c$ so that the matrix is orthogonal.

Alternatively, the fact that $Au=v$, i.e.
$$A\begin{pmatrix}1\\1\\1\end{pmatrix}=\begin{pmatrix}0\\0\\\sqrt{3}\end{pmatrix},$$
shows that the first two rows of $A$ are orthogonal to $u$. Because $A$ is an orthogonal matrix, it follows that the third row is orthogonal to the first two, so the third row is a scalar multiple of $u$. Its inner product with $u$ equals $\sqrt{3}$, so the third row must equal $\frac{u}{\sqrt{3}}$.
Can you proceed in this way with $Aw=-w$ to determine $A$?
</s>