[BUG] add consistency to_numeric on empty list
to_numeric should work similarly on empty lists for
downcast=unsigned/float/integer

Addresses: GH32493

- [x] closes #32493
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

thanks @mikekutzma 