BUG: DataFrame with Int64 columns casts to float64 with .max()/.min()
- [ ] closes #32651
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

It looks like #25777 needs to be fixed first, so closing to clear queue. @jorisvandenbossche @jreback This PR in the current state would close #32651 

BUT the issue would persist for DataFrames with non-numeric columns.

I see the benefit of the changes here is consistency of the code path taken regardless of numeric_only parameter for numeric only DataFrames. I can't find an issue for this, but could potentially concoct one.

There is overlap here with #32867 and #32867 would likely close #32651 anyway.

I'll leave this as draft for now, or could close to clear queue if preferred.

in the meantime, I assume investigating #21020 would also benefit #32867.

There's also an issue with PeriodArray raising `TypeError: mean is not implemented for PeriodArray since the meaning is ambiguous` when dispatching to the EA which is currently ignored with DataFrame.mean containing Period column that I could also look into.

```
>>> df = pd.DataFrame(
...     {
...         "A": np.arange(3),
...         "B": pd.date_range("2016-01-01", periods=3),
...         "C": pd.timedelta_range("1D", periods=3),
...         "D": pd.period_range("2016", periods=3, freq="A"),
...     }
... )
>>> df.mean()
<stdin>:1: FutureWarning: DataFrame.mean and DataFrame.median with numeric_only=None will include datetime64 and datetime64tz columns in a futu
re version.
A                  1
C    2 days 00:00:00
dtype: object
>>>
```

I guess we want to maintain the same behaviour.  or is inconsistent behaviour not desirable.

```
>>> df = pd.DataFrame(
...     {
...         "D": pd.period_range("2016", periods=3, freq="A"),
...     }
... )
>>>
>>> df.mean()
Series([], dtype: float64)
>>>
>>> df.D.mean()
Traceback (most recent call last):
...
TypeError: mean is not implemented for PeriodArray since the meaning is ambiguous.  An alternative is obj.to_timestamp(how='start').mean()
>>>
```
  I just wanted to comment that this overlaps with https://github.com/pandas-dev/pandas/pull/32867, as you also just mentioned. 

Regarding mean of periods, see also https://github.com/pandas-dev/pandas/pull/33758 can you merge master and update to comments I think we should rather fix this in #32867 (the test could still be useful though, so that could be merged here as an xfail, or I can move it to that PR) > I think we should rather fix this in #32867 (the test could still be useful though, so that could be merged here as an xfail, or I can move it to that PR)

closing in favour of #32867

