PERF: perform a unique intersection with IntervalIndex if at least one side is unique
Resolves https://github.com/pandas-dev/pandas/issues/26709
```
       before           after         ratio
     [3ff4f38f]       [30bd5369]
     <datetime_iter~1>
-       749±100μs         365±70μs     0.49  index_object.IntervalIndexMethod.time_intersection_duplicate(1000)
-        59.6±3ms       8.04±0.2ms     0.13  index_object.IntervalIndexMethod.time_intersection_duplicate(100000)
           failed       2.50±0.01s      n/a  index_object.IntervalIndexMethod.time_intersection_duplicate(10000000)
```

- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

@qwhelan Thanks for your quick respond.

Maybe adding check for duplicate nan in _either_ side can resolve.
` if self.isna().sum() > 1 or other.isna().sum() > 1:` can you merge master & try @makbigc suggestion # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=h1) Report
> Merging [#26711](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0f3e8e8bd2118d56c7febe5d6081c5058cb22be7?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26711/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26711      +/-   ##
==========================================
- Coverage   91.71%    91.7%   -0.02%     
==========================================
  Files         178      178              
  Lines       50740    50742       +2     
==========================================
- Hits        46538    46533       -5     
- Misses       4202     4209       +7
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.3% <100%> (-0.01%)` | :arrow_down: |
| #single | `41.21% <0%> (-0.11%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/26711/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `96.11% <100%> (-0.32%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26711/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26711/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.88% <0%> (-0.12%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/26711/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `90.84% <0%> (-0.11%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=footer). Last update [0f3e8e8...66d323b](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=h1) Report
> Merging [#26711](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0f3e8e8bd2118d56c7febe5d6081c5058cb22be7?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/26711/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #26711      +/-   ##
==========================================
- Coverage   91.71%    91.7%   -0.02%     
==========================================
  Files         178      178              
  Lines       50740    50742       +2     
==========================================
- Hits        46538    46533       -5     
- Misses       4202     4209       +7
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.3% <100%> (-0.01%)` | :arrow_down: |
| #single | `41.21% <0%> (-0.11%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/indexes/interval.py](https://codecov.io/gh/pandas-dev/pandas/pull/26711/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9pbnRlcnZhbC5weQ==) | `96.11% <100%> (-0.32%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/26711/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `78.94% <0%> (-10.53%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/26711/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.88% <0%> (-0.12%)` | :arrow_down: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/26711/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `90.84% <0%> (-0.11%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=footer). Last update [0f3e8e8...66d323b](https://codecov.io/gh/pandas-dev/pandas/pull/26711?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 @makbigc Thanks, your suggestion works. The calls to `.is_unique` appear to give us that condition for free, so I've only added `self.isna().sum() <= 1`

@jreback Done and passing thanks @qwhelan and @makbigc 