BUG: REF: Groupby refactor
This is a follow-up, scaled back version of https://github.com/pandas-dev/pandas/pull/46258, that hopefully should be easier to merge.

- [NA, this is small ] closes #xxxx (Replace xxxx with the Github issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I added a merge commit again because the CI run seemed to be failing because it was using an old version of the PR, before I updated the tests? Anyways, this PR seems to be innocent of any failing tests. OK, some of the tests (https://github.com/pandas-dev/pandas/runs/6011207511?check_suite_focus=true) seem to still not be picking up the changed tests and are failing? But some of them have picked up the modified tests and are therefore passing? Not sure what this means, is the CI being weird or does this actually mean I need to adjust the tests more because I'm missing something? @NickCrews - does the one failing test pass for you locally? > @NickCrews - does the one failing test pass for you locally?

Poking into this showed that there was a second `assert` in the test case that I hadn't noticed and updated 🤦 . Looks good locally now, probably CI will pass. little rebase to fix a merge conflict that appeared from the whatsnew changing in main hmm i think this needs to merge master once more
@NickCrews  oh nvm

i was looking at an older version

thanks @NickCrews  Thanks @NickCrews!