PREF: Fix regression from #58984
- [x] closes #59022 (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

```
| Change   | Before [dd87dd3e] <main>   | After [e5a23fb1] <gh59022>   |   Ratio | Benchmark (Parameter)                                       
                               |
|----------|----------------------------|------------------------------|---------|--------------------------------------------------------------------------------------------|
| +        | 24.0±0.7ms                 | 28.8±0.7ms                   |    1.2  | groupby.GroupByCythonAgg.time_frame_agg('float64', 'first')                                |
| +        | 73.8±1ms                   | 85.3±1ms                     |    1.16 | groupby.GroupByCythonAggEaDtypes.time_frame_agg('Int64', 'min')                            |
| +        | 1.53±0.07ms                | 1.68±0.05ms                  |    1.1  | groupby.GroupByMethods.time_dtype_as_group('int16', 'value_counts', 'direct', 1, 'cython') |
| -        | 11.4±0.4ms                 | 10.3±0.2ms                   |    0.91 | groupby.Cumulative.time_frame_transform('int64', 'cummax', False)                          |
| -        | 1.45±0.04ms                | 1.31±0.05ms                  |    0.9  | groupby.GroupByMethods.time_dtype_as_field('int', 'quantile', 'direct', 1, 'cython')       |
| -        | 67.9±3μs                   | 60.9±1μs                     |    0.9  | groupby.GroupByMethods.time_dtype_as_group('int', 'cumsum', 'direct', 1, 'cython')         |
| -        | 26.1±0.3ms                 | 23.1±0.2ms                   |    0.89 | groupby.Cumulative.time_frame_transform('Float64', 'cumsum', False)                        |
| -        | 74.0±2μs                   | 65.9±1μs                     |    0.89 | groupby.GroupByMethods.time_dtype_as_field('uint', 'sem', 'direct', 1, 'cython')           |
| -        | 2.34±0.1ms                 | 2.07±0.01ms                  |    0.89 | groupby.GroupByMethods.time_dtype_as_group('float', 'quantile', 'direct', 1, 'cython')     |
| -        | 660±40μs                   | 575±10μs                     |    0.87 | groupby.GroupByMethods.time_dtype_as_field('int16', 'pct_change', 'direct', 1, 'cython')   |
| -        | 23.1±0.1ms                 | 20.0±0.4ms                   |    0.86 | groupby.Cumulative.time_frame_transform('Int64', 'cumsum', False)                          |
| -        | 90.3±10ms                  | 75.6±1ms                     |    0.84 | groupby.GroupByCythonAggEaDtypes.time_frame_agg('Int64', 'sum')                            |
| -        | 742±40μs                   | 590±20μs                     |    0.8  | groupby.GroupByMethods.time_dtype_as_field('uint', 'pct_change', 'direct', 1, 'cython')    |
| -        | 104±30μs                   | 81.4±1μs                     |    0.78 | groupby.GroupByMethods.time_dtype_as_field('uint', 'median', 'direct', 1, 'cython')        |
| -        | 1.30±0.5ms                 | 548±30μs                     |    0.42 | groupby.GroupByMethods.time_dtype_as_field('uint', 'min', 'direct', 1, 'numba')            |
``` Thanks @luke396 