<s>[INST] How to find two variables $a,b \in {\bf Z}$ that the matrix $A$ is orthogonal I have to find two variables $a,b \in {\bf Z}$ that the given $n \times n$ matrix A becomes orthogonal.
\begin{equation*}
A =
\begin{pmatrix}
 1&2 \\ 
 a&b 
\end{pmatrix}
\end{equation*}
I know that a $n \times n$ matrix is called orthogonal if $A^TA$ $=$ $id$ which means:
\begin{equation*}
  \begin{pmatrix}
   1&a \\ 
   2&b 
  \end{pmatrix}
\cdot
  \begin{pmatrix}
   1&2 \\ 
   a&b 
  \end{pmatrix}
=
  \begin{pmatrix}
   1+a^2&2+ab \\ 
   2+ab&4+b^2 
  \end{pmatrix}
\stackrel{?}{=}
  \begin{pmatrix}
   1&0 \\ 
   0& 1
  \end{pmatrix}
\end{equation*}
 [/INST] You are right. Now solve the system of equation in terms of $a$ and $b$. However, if $b$ is restricted to be an integer, the equation $$4 + b^2 = 1$$ does not have a solution.
</s>