BUG: Allow all int types for merge (GH28870)
- [x] closes #28870
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry
thanks @jjlkant  a side issue, i suspect we don't have asv (performance tests for all of the int indexers) (and its possible since we don't have implementations for some of the non-int64 indexers they may be non-performant).

would you be up to submitting a PR for this?

https://github.com/pandas-dev/pandas/issues/28922