<s>[INST] Is $\int^x \cos \frac1t$ differentiable at zero? From Spivak's Calculus, 4th ed., exc 14-20: 

Let  $$f(x) = \begin{cases} \cos \frac1x, & x\neq 0\\ 0, &x=0.
 \end{cases}$$  Is the function $\int_0^xf$ differentiable at zero?

I'm having trouble with this one; I'm sure I'm just missing something easy.
We seek 
$$\lim_{h\to0} \frac1h \int_0^h \cos\frac1\xi d\xi.$$
A crude estimation tells us that $\frac1h \int_0^h \cos\frac1\xi d\xi$ is between $-1$ and $1$.
Intuitively, it seems that $\int_0^h\cos\frac1\xi d\xi$ is going to zero like $h$.
I considered changing variables (although Spivak has not yet introduced this technique, so it shouldn't be necessary) to 
$$\frac1h \int_{1/h}^\infty \frac{1}{u^2} \cos u,$$
but that didn't really help.
 [/INST] Without that substitution, we can note that
$$g(x) = \begin{cases} -x^2\sin \frac{1}{x} &, x \neq 0 \\ \quad 0 &, x = 0\end{cases}$$
is differentiable everywhere, with $g'(0) = 0$ and
$$g'(x) = \cos \frac{1}{x} -2x\sin\frac{1}{x}$$
for $x\neq 0$. So
$$\int_0^h \cos \frac{1}{\xi}\,d\xi = \int_0^h g'(\xi) + 2\xi\sin \frac{1}{\xi}\,d\xi = g(h) + 2\int_0^h \xi\sin \frac{1}{\xi}\,d\xi.$$
Now it is easy to see that
$$\frac{1}{h} \int_0^h \cos \frac{1}{\xi}\,d\xi = \frac{g(h)}{h} + \frac{2}{h}\int_0^h \xi\sin \frac{1}{\xi}\,d\xi$$
tends to $0$ for $h\to 0$ by the mean value theorem for integrals, since the integrand of the remaining integral is bounded by $\lvert h\rvert$ in absolute value.

The substitution $u = \frac{1}{\xi}$ leads to
$$\frac{1}{h} \int_{1/h}^\infty \frac{\cos u}{u^2}\,du,$$
and we can get further with an integration by parts:
$$\begin{align}
\int_a^b \frac{\cos u}{u^2}\,du &= \left[\frac{\sin u}{u^2}\right]_a^b + 2\int_a^b \frac{\sin u}{u^3}\,du\\
&= \frac{\sin b}{b^2} - \frac{\sin a}{a^2} + 2\int_a^b \frac{\sin u}{u^3}\,du.\tag{1}
\end{align}$$
We can estimate the remaining integral
$$\left\lvert 2\int_a^b \frac{\sin u}{u^3}\,du\right\rvert \leqslant 2\int_a^\infty \frac{du}{u^3} = \frac{1}{a^2}.$$
So we find
$$\left\lvert \frac{1}{h} \int_0^h \cos \frac{1}{\xi} \,d\xi\right\rvert \leqslant \frac{1}{h}\left(\frac{2}{1/h^2} + \frac{1}{1/h^2}\right) \leqslant 3h$$
for $h > 0$. The case $h < 0$ is symmetric, so we see that the integral $F$ of $f$ is differentiable in $0$ with $F'(0) = 0$.
</s>