BUG: transform and filter misbehave when grouping on categorical data
Fixes GH #9921
closes #9700 

That small change fixed most of the performance difference:

```
-------------------------------------------------------------------------------
Test name                                    | head[ms] | base[ms] |  ratio   |
-------------------------------------------------------------------------------
groupby_transform_series2                    | 124.1800 | 124.5937 |   0.9967 |
groupby_transform_series                     |  19.8309 |  19.8836 |   0.9974 |
groupby_transform                            | 151.5836 | 151.8770 |   0.9981 |
groupby_transform_ufunc                      | 112.3510 | 110.2444 |   1.0191 |
groupby_transform_multi_key4                 | 143.6467 | 137.7186 |   1.0430 |
groupby_transform_multi_key3                 | 802.0917 | 753.2163 |   1.0649 |
groupby_transform_multi_key2                 |  49.4637 |  46.3487 |   1.0672 |
groupby_transform_multi_key1                 |  73.4100 |  68.7597 |   1.0676 |
```
 lgtm. pls squash, and ping when green.
 Conditional is cleaned up, squashed, and tests are green.
 @evanpw thanks!
