<s>[INST] Small-angle approximation of $ \frac{\sin^2 x}{x^2 \sqrt{1-\frac{\sin^2 x}{3}}} $ I need to show the following:
$$ \frac{\sin^2 x}{x^2 \sqrt{1-\frac{\sin^2 x}{3}}} \approx 1-\frac{x^2}{6} $$ when $ x $ is small.
I think this problem is trickier than most other questions like it because in the original source there is comment saying "if you got $ 1+\frac{x^2}{6} $ [what I got] then think again!". My attempt was:
When $ x $ is small, $ \sin x \approx x $ so
$$ \frac{\sin^2{x}}{x^2 \sqrt{1-\frac{\sin^2 x}{3}}} = \frac{x^2}{x^2 \sqrt{1-\frac{x^2}{3}}} = \left ( 1 - \frac{x^2}{3} \right )^{-\frac{1}{2}} $$
Then using the binomial series approximation,
$$ \left ( 1 - \frac{x^2}{3} \right )^{-\frac{1}{2}} \approx 1 - \frac{1}{2}\left ( -\frac{x^2}{3} \right ) + ... = 1 + \frac{x^2}{6} $$
...and so it looks like I've fallen into whatever trap the question set.
Where is my error?
 [/INST] Suppose for simplicity you had two polynomials $P(z) = 1+ z + z^2 + 4z^4 + 7z^5$ and $Q(z) = 1 + 2z + 3z^2 + 4z^3$, and I asked you to calculate the product $P(z)Q(z)$... but not the entire thing. Suppose I only want the terms up to quadratic term; i.e if we write
\begin{align}
P(z)Q(z) &= a_0 + a_1z + a_2z^2 + a_3z^3 + a_4z^4 + a_5z^5 +a_6z^6 + a_7z^7 +a_8z^8
\end{align}
then I'm asking you to find the coefficients $a_0,a_1,a_2$ (but for now, let's just say for some reason I'm interested in what happens when $|z|$ is very small up to an accuracy of quadratic order, so I don't really care
about the rest of the terms ). Well, we just multiply everything out:
\begin{align}
P(z)Q(z) &= (1+ z + z^2 + 4z^4 + 7z^5)(1 + 2z + 3z^2 + 4z^3) \\
&= 1 + (1\cdot 2z + z \cdot 1) + (1\cdot 3z^2 + z\cdot 2z + z^2 \cdot 1) \\
&+ \text{(terms involving $z^3$ or higher, which I don't care about for now)} \\
&= 1 + 3z + 6z^2 + O(z^3) 
\end{align}
In other words, because in my final product, I'm only interested in calculating up to the quadratic term, I can simply truncate the polynomials $P$ and $Q$ to quadratic order, and then multiply them out (and then again only keep terms up to quadratic order):
\begin{align}
P(z)Q(z) &= (1 + z + z^2 + \cdots)(1 + 2z + 3z^2 + \cdots) \\
&= 1 + 3z + 6z^2 + O(z^3)
\end{align}
Again, because I'm only interested up to quadratic order, there's no need for me to keep any terms beyond that for $P(z)$ and $Q(z)$, because if I approximate $P(z) \approx 1+ z + z^2 + \color{red}{4z^4}$ (i.e I keep the $4^{th}$ order term) and I multiply with $Q(z) = 1+2z+3z^2 + 4z^3$, then the red term multiplied with anything in $Q(z)$ will yield terms which are $4^{th}$ order or higher (which I don't care about).

But what you should not do is truncate $P(z)$ and $Q(z)$ up to linear order, and say that
\begin{align}
P(z)Q(z) & \approx (1+z)(1+2z) = 1 + 3z + 2z^2
\end{align}
Because in this way, you're missing out other second order contributions (by multiplying constant term of $P$ with quadratic term of $Q$ and vice-versa).


This is how you know how many terms you need to use in your approximation. In your case, you want to approximate
\begin{align}
f(x) &= \dfrac{\sin^2x}{x^2\sqrt{1 - \frac{\sin^2x}{3}}}
\end{align}
up to $2^{nd}$ order. So, what do we do? We write things as a product first:
\begin{align}
f(x) &= \left(\dfrac{\sin x}{x}\right)\cdot\left(\dfrac{\sin x}{x}\right) \cdot \left(\dfrac{1}{\sqrt{1- \frac{\sin^2x}{3}}}\right)\tag{$1$}
\end{align}
Now, we have to expand each bracketed term up to atleast second order in $x$ and then multiply the result together. First:
\begin{align}
\dfrac{\sin x}{x} &= \dfrac{x - \dfrac{x^3}{6} + O(x^4)}{x} = 1 - \dfrac{x^2}{6} + O(x^3) \tag{$2$}
\end{align}
Next, we recall that
\begin{align}
\dfrac{1}{\sqrt{1-z}} &= 1+ \dfrac{z}{2} + \dfrac{3z^2}{8} + O(z^3)
\end{align}
Now, plug in $z= \frac{\sin^2x}{3} = x + O(x^3)$, to get
\begin{align}
\dfrac{1}{\sqrt{1-\frac{\sin^2x}{3}}} &= 1+ \dfrac{1}{2}\left(\dfrac{\sin^2x}{3}\right) + \dfrac{3}{8}\left(\frac{\sin^2x}{3}\right)^2 + O((\sin^2 x)^3) \\
&= 1 + \dfrac{1}{2}\left(\dfrac{x^2 + O(x^4)}{3}\right) + O(x^4) + O(x^6) \\
&= 1 + \dfrac{1}{6}x^2 + O(x^4)\tag{$3$},
\end{align}
where in the second line, hopefully it's clear how I got the various terms: for example $\sin x = x + O(x^3)$, so  $\left(\frac{\sin^2x}{3}\right)^2 = \frac{1}{9}\sin^4x = \frac{1}{9} (x + O(x^3))^4 = O(x^4)$. Therefore, the final answer is obtained by plugging $(2)$ and $(3)$ into $(1)$ :
\begin{align}
f(x) &= \left(1 - \dfrac{x^2}{6} + O(x^3)\right)^2 \cdot \left(1 + \dfrac{1}{6}x^2 + O(x^4)\right) \\
&= 1 - \dfrac{x^2}{6} + O(x^4)
\end{align}

Long story short, if your end goal is to calculate up to second order, then at each stage of your algebra make sure you're keeping terms atleast up to $x^2$.
</s>