Backport PR #46602 on branch 1.4.x (CI/TST: Make test_vector_resize more deterministic)
Backport PR #46602

(cherry picked from commit bea02f300bdf54d86aded7d5c4461dc340ddb929)

- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).

Additionally migrates all the refactoring work done in `test_hashtables.py`

hmm i don't think the backport is clean here