<s>[INST] What is the maximum value of this trigonometric expression What is the maximum value of the expression
$1/(\sin^2 \theta + 3\sin\theta \cos\theta+ 5\cos^2 \theta$).
I tried reducing the expression to $1/(1 + 3\sin\theta$ $\cos\theta + 4\cos^2 \theta)$. 
How do I proceed from here?
 [/INST] There are as usual many approaches.  We describe one that uses pure trigonometry (and in particular no calculus), in line with the tag on the question.
Trigonometry can be bypassed in favour of algebra, as we show in the second part of this answer. But it would not be a good idea to omit the trigonometric approach altogether, since in the process we describe an idea that is useful in Physics, Electrical Engineering, and elsewhere.
A solution via trigonometry: In order to have less messy expressions, we will mainly study
$$\sin^2\theta+3\sin\theta\cos\theta+5\cos^2\theta.$$
It looks sensible enough to use $\sin^2\theta=1-\cos^2\theta$ to rewrite our expression as
$$1+3\sin\theta\cos\theta+4\cos^2\theta.$$
Using the trigonometric identities $2\sin\theta\cos\theta=\sin 2\theta$ and $\cos 2\theta=2\cos^2\theta-1$,  we can then rewrite our expression as
$$3+\frac{3}{2}\sin 2\theta +2\cos 2\theta.$$
Next comes the idea which is useful elsewhere. We express $\frac{3}{2}\sin2\theta+2\cos 2\theta$ as $k \sin(2\theta+\alpha)$ for suitable $k$ and $\alpha$.
The idea is to use the identity
$$\sin(2\theta +\alpha)=(\sin2\theta)(\cos\alpha)+(\cos2\theta)(\sin\alpha).$$
If our expression is to be equal to $k\sin(2\theta+\alpha)$,
we need to have $(3/2)/k=\cos\alpha$ and $2/k=\sin\alpha$. This means that $k=\sqrt{(3/2)^2+2^2}$.  It turns out that $k$ is the nice number $5/2$ (thank you, problem designer). So we are looking at
$$3+\frac{5}{2}\left(\frac{3}{5}\sin 2\theta +\frac{4}{5}\cos 2\theta\right).$$
This is
$$3+\frac{5}{2}\sin(2\theta+\alpha),$$
where $\alpha$ is the angle whose cosine is $3/5$ and whose sine is $4/5$. Note by the way that our expression is always positive, since $\sin(2\theta+\alpha)$ can never be less than $-1$.
The above expression is smallest when $\sin(2\theta+\alpha)=-1$. This is obviously achievable by letting $2\theta=3\pi/2-\alpha$.
So the minimum  value of our expression is $1/2$.
The maximum value of the original function is therefore $2$.  If we wanted the minimum value of the original function, that is also easily found.
Comment: The method that we used to get to the expression $k\sin(2\theta+\alpha)$ can be used to express $a\sin\phi +b\cos\phi$, where $a$ and $b$ are constants, as $k\sin(\phi + \delta)$, for suitable $k$ and $\delta$.
A solution via algebra: Let $x=\sin\theta$ and $y=\cos\theta$.  We want to minimize $x^2+3xy+5y^2$ subject to the condition $x^2+y^2=1$.
To do this we examine the values of $k$ for which the curves $x^2+3xy+5y^2=k$ and $x^2+y^2=1$ kiss.  (From the geometry we can see that at the minimum the two curves will be tangent, so have double point of intersection. That will also happen at the maximum.)
For the minimum, exactly one of $x$ and $y$ will be negative, it doesn't matter which. So let $x=\sqrt{1-y^2}$. Substitute for $x$ in $x^2+ 3xy+4y^2=k$.  We obtain $4y^2-(k-1)=-3y\sqrt{1+y^2}$.  Square both sides and simplify.  We arrive at the equation
$$25y^4-(8k+1)y^2 +(k-1)^2=0.$$
For $y^2$ to be a double root of this equation, the discriminant must be $0$.  So we obtain the equation $(8k+1)^2=100(k-1)^2$, and thus
$8k+1=\pm10(k-1)$.  There is an extraneous root which gives us the maximum. The relevant $k$ is $1/2$. Thus the minimum value of  $x^2+3xy+5y^2$ subject to $x^2+y^2=1$ is $1/2$, so the maximum value of the function in the post is $2$.
</s>