BUG: Fix inconsistent returned objects when applying groupby aggregations
- [x] closes #61503 
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] ~Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.~
- [ ] ~Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.~

Thanks @arthurlw, can you add a test that checks that the bug is indeed fixed. And a release note as well. Thank you!  Thanks @arthurlw  Thanks @arthurlw
@mroeschke : is there any chance this could be added to an earlier release?
I see there is a 2.3.1 milestone for example.