<s>[INST] What is the derivative of $\mathbf{a}^TX^2\mathbf{a}$ with respect to the symmetric matrix $X$? Given a constant vector $\mathbf{a}\in{\rm I\!R}^n$ and a real symmetric matrix $X\in{\rm I\!R}^{n\times n}$, what is the derivative of $\mathbf{a}^T X^2 \mathbf{a}$ with respect to $X$?

I tried a simple example using $n=2$, with the following vector and matrix :
$$
\mathbf{a} = \begin{bmatrix}a\\b\end{bmatrix} \qquad X = \begin{bmatrix} x & z \\ z & y \end{bmatrix}
$$ 
What we want to differentiate is
$$ 
\mathbf{a}^T X^2 \mathbf{a} = a^2 (a^2 + z^2) + 2 ab (xz+yz) + b^2 (z^2 + y^2) 
$$ 
Which gives the following result
$$ 
\frac{\partial \mathbf{a}^T X^2 \mathbf{a}}{\partial X} = 
\begin{bmatrix}
\frac{\partial \mathbf{a}^T X^2 \mathbf{a}}{\partial x} & \frac{\partial \mathbf{a}^T X^2 \mathbf{a}}{\partial z} \\
\frac{\partial \mathbf{a}^T X^2 \mathbf{a}}{\partial z} & \frac{\partial \mathbf{a}^T X^2 \mathbf{a}}{\partial y}
\end{bmatrix} 
=
2
\begin{bmatrix}
a^2 x + abz &  a^2z + abx + aby + b^2z \\
a^2z + abx + aby + b^2z & b^2 y + abz
\end{bmatrix} 
$$ 
I tried to re-write this to end up with something meaningful, but I could only write 
$$ 
\frac{\partial \mathbf{a}^T X^2 \mathbf{a}}{\partial X} = 2X\mathbf{a}\mathbf{a}^T + 2 
\begin{bmatrix}
0 & a^2 z + ab y \\
abx + b^2z & 0
\end{bmatrix}
$$
and I do not know what to do with the matrix on the right...

What I would like is a valid expression for any $n > 1$ involving only $X$ and $\mathbf{a}$. 
 [/INST] Hint
Fréchet derivative of $f(X) =\mathbf{a}^TX^2\mathbf{a}$ is given by
$$\partial_{X_0}f(h) = \mathbf{a}^TX_0 h\mathbf{a} + \mathbf{a}^Th X_0 \mathbf{a}$$
</s>