CI: Try running single_cpu/not single_cpu tests together
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

If I did this right, should get us from 20 jobs to around 14 jobs.

This'll probably slow down the current jobs by ~10 minutes (but they already take 1 hr), but we'll save at least the 5 minutes of compiling pandas on the single_cpu jobs + whatever it takes to setup the conda envs there.

Hopefully the shorter queue times during peak periods makes up for the extra time.
This doesn't quite work yet (need to figure out how to propagate the PATTERN variable). I would recommend bumping up the `timeout-minutes` since 1 job has to run more sets of tests now > I would recommend bumping up the `timeout-minutes` since 1 job has to run more sets of tests now

It's already 3 hours which should be fine (all tests seem to take about 1ish hour). Looks like on the latest commit that the single cpu task isn't running Nice cleanup! Thanks @lithomas1. Hoping to follow up with https://github.com/pandas-dev/pandas/pull/52211 to reduce the number of queued workflows even further 