PERF: Improve performance when accessing GroupBy.groups
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.


```
df = DataFrame(np.random.randint(1, 100, (5_000_000, 5)))
df.groupby([0, 1]).groups

# main
%timeit df.groupby([0, 1]).groups
1.68 s ± 7.52 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
# pr
%timeit df.groupby([0, 1]).groups
504 ms ± 9.24 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```
Thanks @phofl 