Revert "REF: back IntervalArray by a single ndarray (#37047)"
This reverts commit 9cb372376fc78aa66e2559de919592007b74cfaa.

See discussion in https://github.com/pandas-dev/pandas/pull/37047/

@jbrockmendel can you give this a check? (there were already several conflicts)
I'll take a look, tentatively looks OK.

By funny coincidence, I _just_ got the first of the promised perf improvements working. @jbrockmendel ok here? looks fine >  I just got the first of the promised perf improvements working.

Do you have a branch for it that you can push? I am curious to see whether the principles of the speed-up also couldn't be applied to the "2 1D-arrays" model > Do you have a branch for it that you can push? I am curious to see whether the principles of the speed-up also couldn't be applied to the "2 1D-arrays" model

shortly, i hope.  The approach is to do `self._combined.view("complex128")` and then do set operations on that to avoid casting to object. I can imagine that an additional copy to create the combined array could still be worth it compared to casting to object