BUG: Index.difference not always returning a unique set of values
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/v2.2.0.rst` file if fixing a bug or adding a new feature.

`Index.difference` has two short-circuit paths which currently differ in behavior from the main path in that they do not return a unique set of values when duplicates exist:

```
In [1]: import pandas as pd

In [2]: idx = pd.Index([1, 1])

In [3]: idx.difference([2])
Out[3]: Index([1], dtype='int64')      <- main path: unique

In [4]: idx.difference([])
Out[4]: Index([1, 1], dtype='int64')   <- short circuit path when right is empty: non-unique

In [5]: idx.difference([True])
Out[5]: Index([1, 1], dtype='int64')   <- short circuit path when right is considered "non-comparable": non-unique
```
Thanks @lukemanley 