PERF: IntervalArray.unique, IntervalIndex.intersection
- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [ ] whatsnew entry

Partially address perf hit discussed in #41929 

```
import numpy as np
from pandas import *

N = 10**5

left = np.append(np.arange(N), np.array(0))
right = np.append(np.arange(1, N + 1), np.array(1))
intv = IntervalIndex.from_arrays(left, right)

intv2 = IntervalIndex.from_arrays(left + 1, right + 1)

%timeit intv.intersection(intv2)
122 ms ± 2.34 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- PR
263 ms ± 5.98 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  # <- master
```

xref #36482
@meeseeksdev backport 1.3.x Something went wrong ... Please have  a look at my logs.