PERF: Improve performance in read_csv with numeric index col
- [x] closes #44158
- [ ] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

The isin was calling the np.in1d and was checking against a bunch of strings causing the performance issues
the benchmark runner failed, not really sure why I think I have used the wrong base class, changed it greenish now thanks @phofl very nice
keep em coming!