<s>[INST] How many solutions does this have, modulo $p$? $p$ is a prime number, and $p<14$. Furthermore, $x,y\in\mathbb{Z}$ and $0\leq x<13$ and $0\leq y<13$. Let $a,b,c$ denote unknown variables. How many solutions does the following system of equations have, modulo $p$? (that is, two solutions are the same if they are congruent):
\begin{cases}
a+b+c=1\\
a+2b+(x+1)c=y+1\\
a+3b+(x^2+2x+2)c=3y+1
\end{cases}
 [/INST] You have to compute the determinant:
$$\left|\begin{array}{ccc}
1&1&1\\
1&2&x+1\\
1&3&x^2+2x+2
\end{array}
\right|$$
It is
$$2x^2+4x+4-3x-3-x^2-2x-2+x+1+3-2=x^2+1$$
which is not $0$ for $p\in\{3,7,11\}$
For $p=2$, $p=5$ and $p=13$ the determinant can be $0$. Namely:


*

*If $p=2$, when $x$ is odd

*If $p=5$, when $x\in\{2,3,7,8,12,13\}$

*If $p=13$, when $x=5$ or $x=8$.


In all other cases, the determinant is not $0$ and there is only one solution. To consider the other possibilities, let's apply Gauss-Jordan:
$$\left(\begin{array}{ccc|c}
1&1&1&1\\
1&2&x+1&y+1\\
1&3&x^2+2x+2&3y+1
\end{array}
\right)$$
$$\left(\begin{array}{ccc|c}
1&1&1&1\\
0&1&x&y\\
0&2&x^2+2x+1&3y
\end{array}
\right)$$
$$\left(\begin{array}{ccc|c}
1&1&1&1\\
0&1&x&y\\
0&0&x^2+1&y
\end{array}
\right)$$
We see that the system has no solution when $p\mid x^2+1$ but $p\nmid y$. When $p\mid x^2+1$ and $p\mid y$, the matrix has rank $2$. Therefore, the set of solutions has dimension $3-2=1$. Since the filed is $\Bbb F_p$ which has $p$ elements, there are $p$ solutions in this case.
</s>