<s>[INST] Proof of the second symmetric derivative Prove that if $f''(a)$ exists, then $$f''(a)=\lim_{h\to0}\frac{f(a+h)+f(a-h)-2f(a)}{h^{2}}.$$
I really have no idea on this one. Am I supposed to apply the mean value theorem?
 [/INST] Let us make the problem more general trying to approximate the second derivative based on three points $x+ah$, $x+bh$, $x+ch$.
By Taylor, we have $$f(x+kh)=f(x)+h k f'(x)+\frac{1}{2} h^2 k^2 f''(x)+\frac{1}{6} h^3 k^3
   f^{(3)}(x)+\frac{1}{24} h^4 k^4
   f^{(4)}(x)+O\left(h^5\right)$$ Now, consider $$F=Af(x+ah)+Bf(x+bh)+Cf(x+ch)$$ and apply the above formula for each of the terms and group terms; you should get
$$F=f(x) (A+B+C)+h f'(x) (a A+b B+c C)+\frac{1}{2} h^2 f''(x) \left(a^2 A+b^2 B+c^2
   C\right)+\frac{1}{6} h^3 f^{(3)}(x) \left(a^3 A+b^3 B+c^3
   C\right)+\frac{1}{24} h^4 f^{(4)}(x) \left(a^4 A+b^4 B+c^4 C\right)+O\left(h^5\right)$$ What we then want is to only get the term with $f''(x)$; so we need to cancel the previous terms. Then, this implies $$A+B+C=0\tag 1$$ $$a A+b B+c C=0\tag 2$$ $$\frac{1}{2} \left(a^2 A+b^2 B+c^2
   C\right)=1\tag 3$$ So, three linear equations in $A,B,C$ to be solved. This gives $$A=\frac{2}{(a-b) (a-c)}\qquad B=\frac{2}{(b-a) (b-c)}\qquad C=\frac{2}{(c-a) (c-b)}$$ For your case $a=1,b-1,c=0$ which gives $A=1,B=1,C=-2$.
So, we have $$f(x+h)+f(x-h)-2f(x)=h^2 f''(x)+\frac{1}{12} h^4 f^{(4)}(x)+O\left(h^5\right)$$
The same procedure can apply to the derivative of any order.
</s>