TST: Consolidate tests that raise in groupby
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Part of #50133. Continue the work of @rhshadrach started in #50404 for the groupby consolidation.
Thanks for the review @rhshadrach. 

I updated the tests, I believe now it should cover all the cases you mentioned (and more). Wondering if 28k tests for a single file isn't to much though ?

Anyway, I think we are still missing some cases like grouping on the index or grouping on a dtype other than int or category. Not sure if you think it is necessary here? On my machine, I get :
==== 28501 passed, 420 skipped in 113.16s (0:01:53) ====

Also, do you know why the CI pipeline is failing ? I see some failed tests in pandas/tests, but I don't see the connection with this PR ? Done. The new runtime on my machine is : 
===== 10561 passed in 44.23s ===== Thanks @albuzenet - great work! Thanks for the review @rhshadrach !