BUG: downcast = 'unsigend' on 0 would would not downcast to unsigned.
Continuation of #14412
- [ ] closes #14401
- [ ] tests added / passed
- [ ] passes `git diff upstream/master | flake8 --diff`
- [ ] whatsnew entry

Once again I had some very bizarre issues with git. Hope this does it this time.

@verhalenn For future reference, it is normally never needed to open a new PR to fix git issues. You can just clean it up locally and force push.

BTW, you have some failing tests.
 No when I forced pushed it it closed the pull request for some reason. Usually that would happen if I were to delete the branch and then force push but that wasn't the case. I'll look into the error.
 ## [Current coverage](https://codecov.io/gh/pandas-dev/pandas/pull/14472?src=pr) is 85.25% (diff: 100%)

> No coverage report found for **master** at 2e77536.
> 
> Powered by [Codecov](https://codecov.io?src=pr). Last update [2e77536...8db827a](https://codecov.io/gh/pandas-dev/pandas/compare/2e77536bdf90ef20fefd4eab751447918e07668f...8db827a299f38dc2e3b8bf8d749b33ec60d53661?src=pr)
 thanks!
 @jreback : I think this commit is breaking `master` right now.
 reverted: https://github.com/pandas-dev/pandas/commit/d1d75d7fcb6b5d090cf96d328d0136c3959a82b8

@verhalenn if you can re-submit this PR would be great.
 First thing tomorrow morning. I was kind of surprised you closed it when you did. The lint should be an easy fix but the downcast test will take a little extra time. 
