<s>[INST] find the value of ''a'' and ''b'' for which function is always increasing 
Question Let f(x)= x$^{3}+ax^{2}+bx+5sin^{2}x$ be an increasing
function$\forall$ x $\in$$\mathbb{R}$$ Then$

(a) $a^2 -3b-15$<0
(b) $a^2 -3b-15$>0
(c)$a^2 -3b+15<$0
(d)a>0,b>0
My Approach i tried by making perfect square of the derivative
$\left\{ x+\frac{a}{3}\right\} ^{2}$+$\frac{b}{3}$-$\frac{a}{9}-5\geq$0
$\left\{ sin2x\geq-1\right\} $ $\Longrightarrow$a$^{2}$-3b+45$\leq$0
but it doesn't gives the required result
 [/INST] Choice $(a)$ is correct.

For $a,b \in \mathbb{R},\;$let $f:\mathbb{R} \to \mathbb{R}\;$be defined by
$$f(x)= x^3+ax^2+bx+5\sin^2(x)$$

Suppose $a,b$ are such that

$\qquad{\small{\bullet}}\;\;f\;$is an increasing function.

$\qquad{\small{\bullet}}\;\;a^2-3b -15 \ge 0$.

Our goal is to derive a contradiction.

Since $f$ is increasing, it follows that $f'(x) \ge 0$, for all $x \in \mathbb{R}$.

Note that
\begin{align*}
f'(x) &= 3x^2 + 2ax + b + 10\sin(x)\cos(x)\\[4pt]
&= 3x^2 + 2ax + b + 5\sin(2x)\\[4pt]
&=p(x) + 5 + 5\sin(2x)\\[4pt]
\end{align*}
where $p(x) = 3x^2 + 2ax + b - 5$.

By the quadratic formula, $p$ has roots $r_1,r_2$ given by
\begin{align*}
r_1 &= \frac{-a - \sqrt{a^2 + 3b + 15}}{3}\\[4pt]
r_2 &= \frac{-a + \sqrt{a^2 + 3b + 15}}{3}\\[4pt]
\end{align*}
From $a^2 - 3b-15 \ge 0$, we get $a^2 - 3b + 15 \ge 30$.

It follows that $r_1,r_2$ are distinct real numbers, and $p(x) < 0$, for $x \in (r_1,r_2)$.
\begin{align*}
\text{Also,}\;\;r_2 - r_1 &= 
2\left(
\frac
{\sqrt{a^2 + 3b + 15}}
{3}
\right)\\[5pt]
&\ge \frac{2\sqrt{30}}{3}\\[5pt]
& > \frac{2\sqrt{25}}{3}\\[5pt]
&=\frac{10}{3}\\[5pt]
& > \pi\\[4pt]
\end{align*}
Since $r_2-r_1 > \pi$, it follows that for some integer $n$, we have $r_1 < \frac{3\pi}{4} + n\pi < r_2$.

Then letting $t = {\large{\frac{3\pi}{4}}} + n\pi$, we get
\begin{align*}
f'(t) &= p(t) + 5 + 5\sin(2t)\\[4pt]
&=p(t) + 5 + 5\sin\left({\small{\frac{3\pi}{2}}} + 2n\pi\right)\\[4pt]
&=p(t) + 5 + 5\sin\left({\small{\frac{3\pi}{2}}}\right)\\[4pt]
&=p(t) + 5 + 5(-1)\\[4pt]
&=p(t)\\[4pt]
&< 0\\[4pt]
\end{align*}
contradiction.

It follows that choice $(a)$ is correct, as claimed.

If it's given that only one of the choices $(a),(b),(c),(d)$ is correct, then since $(a)$ is correct, we're done.

If we need to show that choice $(a)$ is the only correct choice, then we have more work to do . . .

To invalidate choices $(b)$ and $(d)$, use the values $a=0,\,b=6$.

Then for all $x \in \mathbb{R}$,
\begin{align*}
f'(x)&=3x^2 + 2ax + b + 5\sin(2x)\\[4pt]
&=3x^2 + 6 + 5\sin(2x)\\[4pt]
&\ge 3x^2 + 6 + 5(-1)\\[4pt]
&= 3x^2 +1\\[4pt]
& > 0\\[4pt]
\end{align*}
so $f$ is an increasing function.

But then


*

*Choice $(d)$ fails since$\;a,b\;$are not both positive.$\\[4pt]$

*Choice $(b)$ fails since$\;a^2 - 3b - 15 = (0)^2 - 3(6) - 15 = -33 < 0$.


To invalidate choice $(c)$, let
\begin{align*}
a=&{\small{\frac{\pi}{4}}}\\[4pt]
b=&{\small{\frac{a^2}{3}}}
+\left(
5 - {\small{\frac{1}{10}}}
\right)
\\[4pt]
\end{align*}
and verify that $f'(x) > 0\;$for all $x \in \mathbb{R}$.

The easiest verification is graphical (i.e., graph $f',\;$and see that the graph lies entirely above the $x$-axis). 

It can also be verified algebraically, with a little more work.

I'll omit the algebraic verification for now, but I'll supply the (somewhat messy) details, if requested.

So for the values of $a,b,\;$as given above, $f$ is an increasing function, but
$$a^2 - 3b + 15 = {\small{\frac{3}{10}}} > 0$$
hence choice $(c)$ fails.

Therefore choice $(a)$ is the only correct choice.
</s>