<s>[INST] Evaluate and Simplify $\cos[\cos^{-1}(\frac{3}{5}) + \frac{\pi}{3}]$ I am trying to evaluate and simplify $\cos[\cos^{-1}(\frac{3}{5}) + \frac{\pi}{3}]$.
I am getting $\frac{11}{10}$ but the answer is $\frac{3-4\sqrt{3}}{10}$
My Process:
$\cos[\cos^{-1}(\frac{3}{5}) + \frac{\pi}{3}]$
$\cos[\cos^{-1}(\frac{3}{5})] + \cos(\frac{\pi}{3})$
$(\frac{2}{2}) \cdot \frac{3}{5} + \frac{1}{2} \cdot (\frac{5}{5})$
$\frac{6}{10} + \frac{5}{10}$
$\frac{11}{10}$ 
 [/INST] By compound-angle formula, 
\begin{eqnarray*}
\cos(\cos^{-1}(\frac{3}{5})+\frac{\pi}{3}) & = & \cos\left(\cos^{-1}(\frac{3}{5})\right)\cos\frac{\pi}{3}-\sin\left(\cos^{-1}(\frac{3}{5})\right)\sin(\frac{\pi}{3})\\
 & = & \frac{3}{5}\cdot\frac{1}{2}-\sin\left(\cos^{-1}(\frac{3}{5})\right)\cdot\frac{\sqrt{3}}{2}.
\end{eqnarray*}
To evaluate $\sin\left(\cos^{-1}(\frac{3}{5})\right)$, we let $\theta=\cos^{-1}(\frac{3}{5})$.
Then $\cos\theta=\frac{3}{5}$. Recall that $\sin^{2}\theta+\cos^{2}\theta=1$
and observe that $0<\theta<\frac{\pi}{2}$. Therefore $\sin\theta>0$ and
$\sin\theta$ is given by $\sin\theta=\sqrt{1-\cos^{2}\theta}=\frac{4}{5}$.
It follows that 
\begin{eqnarray*}
\cos(\cos^{-1}(\frac{3}{5})+\frac{\pi}{3}) & = & \frac{3}{5}\cdot\frac{1}{2}-\frac{4}{5}\cdot\frac{\sqrt{3}}{2}\\
 & = & \frac{3}{10}-\frac{2\sqrt{3}}{5}
\end{eqnarray*}
</s>