<s>[INST] How to solve this trig problem? $\sec(\sin^{-1}(-5/13)-\tan^{-1}(4/3))$ Basic trig problem my brother ask me, but I don't know how to do it:
$$\sec(\sin^{-1}(-5/13)-\tan^{-1}(4/3))$$
 [/INST] Since $\sec(x) = 1/\cos(x)$, let's first consider
$$\cos\left[\sin^{-1}(-5/13) - \tan^{-1}(4/3)\right].$$
Using the addition formula for $\cos$, this is equal to
$$\cos(\sin^{-1}(-5/13)) \cos(\tan^{-1}(4/3)) + \sin(\sin^{-1}(-5/13)) \sin(\tan^{-1}(4/3))$$
Now let's work term-by-term.
The key is to recognise that the arguments of the inverse trig functions are part of Pythagorean triples.
First, $\cos(\sin^{-1}(-5/13))$.
If we write $u = \sin^{-1}(-5/13)$, then $\sin u = -5/13$.
These numbers might remind you of the Pythagorean triple $(5,12,13)$, so if
$$-\sin u = \sin(-u) = \frac{\text{OPP}}{\text{HYP}} = \frac{5}{13},$$
then we must have
$$\cos(-u) = \frac{\text{ADJ}}{\text{HYP}} = \frac{12}{13}.$$
Further, since $\cos(-u) = \cos(u)$ for all real $u$, so our first term is $12/13$.
A similar principle works for our terms with $\tan^{-1}(4/3)$, where we use the Pythagorean triple $(3,4,5)$. Write $w = \tan^{-1}(4/3)$. If $\text{OPP}=4$, $\text{ADJ}=3$ and $\text{HYP} = 5$, then
$$\sin(w) = \frac{4}{5} \text{ and } \cos(w) = \frac{3}{5}.$$
Finally, the $\sin(\sin^{-1}(-5/13))$ term simplifies to $-5/13$ (strictly speaking, this depends on our choice of domain for $\sin^{-1}$).
Plugging this all in, we have
$$\cos\left[\sin^{-1}(-5/13) - \tan^{-1}(4/3)\right]
= \frac{12}{13} \cdot \frac{3}{5} + \frac{-5}{13} \cdot \frac{4}{5}
= \frac{16}{65}.
$$
Then we just flip this fraction to get what you actually want:
$$\sec\left[\sin^{-1}(-5/13) - \tan^{-1}(4/3)\right] = \frac{65}{16}.$$
</s>