PERF: slow tests
- [x] closes #44624
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [ ] whatsnew entry

Biggest difference is in test_get_loc_tuple_monotonic_above_size_cutoff going from 12.92 seconds t 4.72, all in improving DatetimeLikeArray.astype.
Thanks!