[ArrayManager] Ensure to store datetimelike data as DatetimeArray/TimedeltaArray (and not ndarray)
Pre-cursor for https://github.com/pandas-dev/pandas/pull/39991

Currently we didn't really check that we were consistently storing datetimelike data as the EA (DatetimeArray, TimedeltaArray) or as ndarrray. Ensuring this in the ArrayManager constructor turns up a few failures.

I *think* it will be the easiest to always store them as EA and not as ndarray (eg for many other operations, we otherwise would wrap them in the EA anyway).
@jbrockmendel more comments here? LGTM cc @jreback @jbrockmendel thanks for the review

I updated https://github.com/pandas-dev/pandas/pull/39991 now based on this