BUG: get_indexer_non_unique with np.nan
- [x] closes #35392
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#code-standards) for how to run them
- [x] whatsnew entry

One of the tests is copied from #35498
I think in order to be consistent, one could use `_lib.hashtable.PyObjectHashTable` (instead of Python's `set`  and `dict`) - it has the "expected" behavior when it comes to nan-floats and can also handle more complex cases (like `(float("nan),)` or `complex(0, float("nan"))`) out-of-the-box.

The downside is, that one has to misuse a table for a set (#39799 would fix that). @realead IIUC what you're suggesting is viable medium-term but likely not short-term?  i.e. is it worth doing a temporary fix like this PR without what you're describing? I must confess, I assumed the solution would be within reach. 

But there are some stumbling blocks:

Panda's `PyObjectHashTable` doesn't perform reference counting

https://github.com/pandas-dev/pandas/blob/master/pandas/_libs/hashtable_class_helper.pxi.in#L1126-L1138

 so using it as set/dict with temporary objects is problematic. 

Also `PyObjectHashTable` maps objects to int64 and not `pyobject`, which is at least a problem for performance. Maybe it is worth to introduce `PandasSet` and `PandasDict` which would have the same handling of nans as pandas algorithm. But until then a fix like that is probably best we can do. > But until then a fix like that is probably best we can do.

Makes sense, thanks for taking a look.

> Maybe it is worth to introduce PandasSet and PandasDict which would have the same handling of nans as pandas algorithm

I'll defer to you on this, as you've mentioned using other khash functionality before.  I'm a bit wary about the increasing build size. closing in favor of #35498