BUG: fix to_datetime to handle int16 and int8
- [x] closes #13451 
- [x] tests added / passed
- [x] passes `git diff upstream/master | flake8 --diff`
- [x] whatsnew entry

Fixes #13451

tests!
 @jreback Hi, I am a bit confused about which file should the tests be written in? There is no separate file that contains tests for pandas.tseries.tools .
 https://github.com/pydata/pandas/blob/master/pandas/tseries/tests/test_timeseries.py#L2453 or you can make another function in the same class
 lgtm (minor comment), pls add a whatsnew entry. ping when green.
 ## [Current coverage](https://codecov.io/gh/pydata/pandas/pull/13464?src=pr) is **84.32%**

> Merging [#13464](https://codecov.io/gh/pydata/pandas/pull/13464?src=pr) into [master](https://codecov.io/gh/pydata/pandas/branch/master?src=pr) will increase coverage by **<.01%**

``` diff
@@             master     #13464   diff @@
==========================================
  Files           138        138          
  Lines         51069      51072     +3   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          43066      43069     +3   
  Misses         8003       8003          
  Partials          0          0          
```

> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [013c2ce...dc4944d](https://codecov.io/gh/pydata/pandas/compare/013c2ce760083e5454552ad44bba8d2f44da2caa...dc4944d120755bc61050cd53e22c9b3d8a672a2e)
 thanks!
