<s>[INST] Substitution in a geometric series: $\sum_{n=0}^{N} x^n = \frac{x^{N+1} -1}{x-1}$ To find a formula for the sums of square of first n natural no, the following method is applied in generating functionology, we know 
$$\sum_{n=0}^{N} x^n = \frac{x^{N+1} -1}{x-1}$$
So we apply the $\left\lbrace xD \right\rbrace^2$  operator on it and put $ x = 1$ to get the corresponding formula. 
The $\left\lbrace xD \right\rbrace^2$ of right side I found as 
$$\frac{(N+1)^2x^{N+1}}{x-1} - \frac{(N+1)x^{N+2}}{(x-1)^2} - \frac{(N+2)x^{N+2}-x}{(x-1)^2} + \frac{2x(x^{N+2} -x)}{(x-1)^3}$$  Here after substituting $x = 1$, however  the denominator are all $0$ .How to eliminate it to get the correct formula i.e $ \frac{N (N+1) (2N +1)} {6}$ 
 [/INST] Hint. I would set $$\epsilon:=x-1$$ then, as $\epsilon \to 0$, use the binomial theorem to get
$$
x^N=(1+\epsilon)^N=1+N\epsilon+\frac{1}{2} N(N-1)\:\epsilon^2+\frac{1}{6} N(N-1)(N-2)\:\epsilon^3+o(\epsilon^3). \tag1
$$ Inserting $(1)$ in your identity gives the sought result.

In fact, by applying $\left\lbrace (1+\epsilon)D_\epsilon \right\rbrace^2$ to
$$
\sum_{n=1}^{N} (1+\epsilon)^n=\frac{(1+\epsilon )^{N+1}-1}{\epsilon }, \quad \epsilon\neq0,\tag2
$$ you get, for $\epsilon\neq0$,
$$
\sum_{n=1}^{N} n^2\times (1+\epsilon)^n=-\frac{2+3 \epsilon +\epsilon ^2-(1+\epsilon )^{N+1} \left(2+\epsilon  \left(3-2 (N+1)+N^2 \epsilon \right)\right)}{\epsilon ^3}.\tag3
$$ Then inserting $(1)$ into $(3)$ leads to
$$
\sum_{n=1}^{N} n^2\times (1+\epsilon)^n=\frac{\frac{N(N+1)(2N +1)}{6}\epsilon^3+o(\epsilon^3)}{\epsilon^3}=\frac{N(N+1)(2N +1)}{6}+o(1)
$$ and letting $\epsilon \to 0$ gives the announced result.
</s>