CLN: dont catch on groupby.mean
The new casting in cython_agg_blocks is specific to a single IntegerArray test case.  We could pretty reasonably move that into maybe_downcast_numeric.  For the moment id rather hold off since I expect other EA cases to show up here.

cc @WillAyd
I think you commented this on my other PR but I have it as a goal to get rid of the block management altogether from groupby. This should be OK in the interim though