<s>[INST] Calculate the determinant $\left|\begin{smallmatrix} a&b&c&d\\ b&a&d&c\\ c&d&a&b\\d&c&b&a\end{smallmatrix}\right|$ Question: Calculate the following determinant
$$A=\begin{vmatrix} a&b&c&d\\ b&a&d&c\\ c&d&a&b\\d&c&b&a\end{vmatrix}$$
Progress: So I apply $R1'=R1+R2+R3+R4$ and get
$$A=(a+b+c+d)\begin{vmatrix} 1&1&1&1\\ b&a&d&c\\ c&d&a&b\\d&c&b&a\end{vmatrix}$$
Then, I apply $C2'=C2-C1,\, C3'=C3-C1$, etc to get
$$A=(a+b+c+d)\begin{vmatrix}1&0&0&0\\b&a-b&d-b&c-b\\c&d-c&a-c&b-c\\d&c-d&b-d&a-d \end{vmatrix}$$
Thus, $$A=(a+b+c+d)\begin{vmatrix}a-b&d-b&c-b\\d-c&a-c&b-c\\c-d&b-d&a-d \end{vmatrix}$$
But now I'm stuck here. I don't really want to expand the whole thing, because it is really messy. Is there any approach that don't require much calculation?
 [/INST] This is a block matrix, you can calculate a determinant of a $2 \times 2$ matrix made of $2 \times 2$ matrices, and then expand that: $$A=\begin{vmatrix} M&N\\ N&M\end{vmatrix}$$ where $$M=\begin{bmatrix} a&b\\ b&a\end{bmatrix}$$ and $$N=\begin{bmatrix} c&d\\ d&c\end{bmatrix}$$
All matrices are symmetric, this is really simple (you don't have to think about commutativity of the matrix products).
$$A=\begin{vmatrix} M&N\\ N&M\end{vmatrix}=|M^2-N^2|$$
$$M^2=\begin{bmatrix} a^2+b^2&2ab\\ 2ab&a^2+b^2\end{bmatrix}$$
$$N^2=\begin{bmatrix} c^2+d^2&2cd\\ 2cd&c^2+d^2\end{bmatrix}$$

$$A=(a^2+b^2-c^2-d^2)^2-4(ab-cd)^2$$

The formula is correct, as can be checked by substituting some numbers.

Edit
A more general case of the determinant of a block matrix. (See here for example).
If $C$ and $D$ commute.
$$|M|= \begin{vmatrix} A & B \\C & D \end{vmatrix}=|AD-BC|$$

Edit 2
Just in case there are any more people claiming this is wrong, here's the check:
Simplify[Det[{{a, b, c, d}, {b, a, d, c}, {c, d, a, b}, {d, c, b, a}}] - (a^2 + b^2 - c^2 - d^2)^2 + 4 (a b - c d)^2]

</s>