<s>[INST] Formal Power Series as Linear Operators Let $t^k$ act as the $k$-th derivative operator on the set of polynomials.  So
$$t^k(x^n)=t^k x^n=(n)_kx^{n-k}$$
where $(n)_k=n(n-1)(n-2)...(n-k+1)$ is the falling factorial.  Then with a formal power series, $f(t)=\sum_{k\ge 0}a_k\frac{t^k}{k!}$, the linear operator $f(t)$ acts as such that
$$f(t)(x^n)=f(t)x^n=\sum_{k=0}^n\binom{n}{k}a_k x^{n-k}$$
Therefore, depending on the coefficients of the power series, we can get some interesting binomial identites.  For example, if $f(t)=e^{yt}$, since the coefficients $a_n=y^n$, we get
$$e^{yt}x^n=\sum_{k=0}^n\binom{n}{k}y^k x^{n-k}=(x+y)^n$$
by linearity, 
$$(e^{yt}-1)x^n=(x+y)^n-x^n=\sum_{k=1}^{n}\binom{n}{k}y^k x^{n-k}$$
and perhaps not as obvious
$$\left(\frac{e^{yt}-1}{t}\right)x^n=\int_{x}^{x+y}u^ndu$$
Now suppose that $f(t)=e^{yt}-1-yt$.  Then
$$(e^{yt}-1-yt)x^n=(x+y)^n-x^n-ynx^{n-1}=\sum_{k=2}^{n}\binom{n}{k}y^k x^{n-k}$$
Obviously there is a nice formed forward difference equation in the previous case that is not happening here.  But there is a relationship with subtracted terms of the binomial expansion.  What i would really like help understanding is whether or not a possible analogous integral representation exists for the following operator:
$$\left(\frac{e^{yt}-1-yt}{t^2}\right)x^n=\left(\sum_{k=0}^\infty\frac{y^{k+2}}{(k+2)(k+1)}\frac{t^k}{k!}\right)x^n=\sum_{k=0}^n\binom{n}{k}\frac{y^{k+2}}{(k+1)(k+2)}x^{n-k}$$  
$$=\sum_{k=0}^n\binom{n+2}{k+2}\frac{y^{k+2}}{(n+1)(n+2)}x^{n-k}=\frac{1}{(n+1)(n+2)}\sum_{k=2}^{n+2}\binom{n+2}{k}y^kx^{n+2-k}$$  
It is not as simple.  Clearly $\frac{d^2}{dx^2}\frac{x^{n+2}}{((n+2)(n+1)}$.  If I integrated below I think the math is correct
$$\int_x^{x+y}{\frac{u^{n+1}}{n+1}}du=\frac{1}{(n+1)(n+2)}\sum_{k=1}^{n+2}\binom{n+2}{k}y^kx^{n+2-k}$$  
Which is really close, but the lower bound on the summation is $1$, not $2$.  Does any one have any insight in how i can fix this, if possible?
 [/INST] Note: OPs calculations are quite ok and it shows the operators are closely related, but different. I don't think there is a necessity to fix anything.
I skimmed through the classic The Umbral Calculus by Steven Roman, but there was no indication that something more is going on regarding OPs question. Another source I've checked without success was The Calculus of Finite Differences by C. Jordan.
It might be helpful to list a few higher powers of the operators under consideration.
In the following I use OPs notation which is precisely the same used by Steven Roman.

Translation operator: $e^{yt}$
Since this operator satisfies:
  \begin{align*}
e^{yt}x^n&=\sum_{k=0}^\infty \frac{y^k}{k!}t^kx^n
=\sum_{k=0}^n \frac{y^k}{k!}(n)_kx^{n-k}=\sum_{k=0}^n\binom{n}{k}y^kx^{n-k}\\
&=(x+y)^n
\end{align*}
  we obtain
  \begin{align*}
\left(e^{yt}\right)^2 x^n=e^{yt}(x+y)^n=(x+2y)^n
\end{align*}
  and in general for $j\geq 1$
  \begin{align*}
e^{jyt}x^n=(x+jy)^n
\end{align*}

Since $x^n, n\geq 0$ form a basis of the vector space of all polynomials $p$ in a single variable $x$ and the translation operator is linear, we obtain
\begin{align*}
e^{jyt}p(x)=p(x+jy)\tag{1}
\end{align*}
hence the name translation operator.

Forward difference operator: $e^{yt}-1$
Here we obtain for polynomials $p$ using (1)
\begin{align*}
\left(e^{yt}-1\right)p(x) = p(x+y)-p(x)
\end{align*}

The next one is

Operator: $\frac{\exp(yt)-1}{t}$
We obtain
  \begin{align*}
\left(\frac{e^{yt}-1}{t}\right)x^n&=\sum_{k=1}^\infty \frac{y^k}{k!}t^{k-1}x^n\\
&=\sum_{k=1}^{n+1}\frac{y^k}{k!}(n)_{k-1}x^{n-(k-1)}\\
&=\frac{1}{n+1}\sum_{k=1}^{n+1}\binom{n+1}{k}y^kx^{n+1-k}\\
&=\frac{1}{n+1}\left((x+y)^{n+1}-x^{n+1}\right)\tag{2}\\
&=\frac{1}{n+1}\int_x^{x+y}u^n\,du
\end{align*}
Similarly we obtain from (2) by linearity
  \begin{align*}
\left(\frac{e^{yt}-1}{t}\right)^2x^n
&=\left(\frac{e^{yt}-1}{t}\right)\frac{1}{n+1}\left((x+y)^{n+1}-x^{n+1}\right)\\
&=\frac{1}{n+1}\left[\frac{1}{n+2}\left((x+2y)^{n+2}-(x+y)^{n+2}\right)\right.\\
&\qquad\qquad\quad\left.-\frac{1}{n+2}\left((x+y)^{n+2}-x^{n+2}\right)\right]\\
&=\frac{1}{(n+1)(n+2)}\left((x+2y)^{n+2}-2(x+y)^{n+1}+x^{n+2}\right)\\
&=\frac{1}{(n+2)_2}\left(\int_{x+y}^{x+2y}u\, du-\int_x^{x+y}u\,du\right)\tag{3}
\end{align*}

Here at (2) and (3) we can see quite nicely how the operator $\frac{\exp(yt)-1}{t}$ is connected with the integral operator. It can be extended to higher powers without too much effort and the relationship with the integral operator looks plausible.

Operator: $\frac{\exp(yt)-1-t}{t^2}$
Now we take  a look at the operator which is on the focus of OP and its generalisation.
\begin{align*}
\left(\frac{e^{yt}-1-t}{t^{2}}\right)x^n
&=\sum_{k=2}^\infty\frac{y^k}{k!}t^{k-2}x^n\\
&=\sum_{k=2}^{n}\frac{y^{k}}{k!}(n)_{k-2}x^{n-(k-2)}\\
&=\frac{1}{(n+2)_2}\sum_{k=2}^n\binom{n+2}{k}y^kx^{n+2-k}\\
&=\frac{1}{(n+2)_2}\left((x+y)^{n+2}-x^{n+2}-nyx^{n+1}\right)
\end{align*}
Comparing the final expression with (3) we do not see a plausible representation via integrals since the term $nyx^{n+1}$ don't provide anything nicely of the form
  \begin{align*}
\text{integrated expression (end point) - integrated expression (starting point)}
\end{align*}

This impression becomes more strongly  when looking at  the general case. We obtain for $j\geq 1$
\begin{align*}
\left(\frac{e^{yt}-1-\frac{t^2}{2}-\cdots-\frac{t^{j-1}}{(j-1)!}}{t^j}\right)x^n
&=\sum_{k=j}^\infty\frac{y^k}{k!}t^{k-j}x^n\\
&=\frac{1}{(n+j)_j}\sum_{k=j}^\infty\binom{n+j}{k}y^kx^{n+j-k}\\
&=\frac{1}{(n+j)_j}\left((x+y)^{n+j}-\sum_{k=0}^{j-1}\binom{n+j}{k}y^kx^{n+j-k}\right)
\end{align*}
</s>