BUG: Groupby not keeping object dtype when infer string is set
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Is this a regression with no related issue? I am working on a build that uses arrow backed strings by default, but this turns up dozens of bugs related to either the new string dtype we added for 2.1 or arrow strings more generally, not opening issues since I am fixing them when I come across them Thanks @phofl 