<s>[INST] How to take the gradient of the quadratic form? It's stated that the gradient of:
$$\frac{1}{2}x^TAx - b^Tx +c$$
is 
$$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$
How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$?
 [/INST] Yet another approach.
We will utilize the following the identities 


*

*Trace and Frobenius product relation $$\left\langle A, B \right\rangle={\rm tr}(A^TB) = A:B$$ or $$\left\langle A^T, B \right\rangle ={\rm tr}(AB) = A^T:B$$

*Cyclic property of Trace/Frobenius product 
\begin{align}
\left\langle A, B C \right\rangle \equiv A:BC 
 &= AC^T:B\\
 &= B^TA:C\\
 &= BC : A\\
 &= {\text{etc.}} \cr
\end{align}
Let $f(x) := \left( \frac{1}{2} x^T A x - b^T x + c \right) $.
We obtain the differential first, and then the gradient subsequently.
\begin{align}
 d\,f(x) 
&=  d\left( \frac{1}{2} x^T A x - b^T x + c \right) \\
&=  d\left( \frac{1}{2} \left( x: A x \right) - \left( b : x \right) + c \right) \\
&=  \frac{1}{2} \left[ \left( dx: A x \right) + \left( x: A dx \right) \right] - \left( b : dx \right)  \\
&=  \frac{1}{2} \left[ \left( A x : dx \right) + \left( A^Tx: dx \right) \right] - \left( b : dx \right)  \\
&=  \frac{1}{2}  \left[ \left(A + A^T  \right)x: dx \right] - \left( b : dx \right)  \\
&=  \left( \frac{1}{2}  \left[ \left(A + A^T  \right)x\right] - b \right): dx \\
\end{align}
Thus the gradient is 
$$ \eqalign {
 \frac { \partial} {\partial x}f(x) &=  \frac{1}{2}  \left[ \left(A + A^T  \right)x\right] - b \\ &=   \frac{1}{2} A^T x +  \frac{1}{2} A x - b . \cr
} $$
</s>