BUG: Don't cast categorical nan to int
- [x] closes #28406
- [x] passes `black pandas`
- [x] tests added / passed
- [x] whatsnew entry

This raises an error when attempting to cast a `Categorical` or `CategoricalIndex` containing `nans` to an integer dtype.  Also had to remove the casting within `get_indexer_non_unique` since this won't always be possible.
FWIW the issue with `test_get_indexer_non_unique ` is here https://github.com/pandas-dev/pandas/blob/master/pandas/core/indexes/base.py#L4717.  If `target` is `Categorical` then the values can't always be cast to the dtype of the categories. should the use_inf_as_na option matter here? > should the use_inf_as_na option matter here?

Wasn't familiar with that option, but it shouldn't affect `numpy` behavior I think? > I think this PR should focus on just NaN handling.
> 
> I'd rather see `Float64Index` properly handle astyping inf values to int (raise), and then do `self.categories.take(self.codes)` with a check if any of the codes is negative, so that we raise.

Ok, so for the purposes of this PR remove the references to `np.inf` in the code and tests? That would be my preference. Did my earlier comment make sense?

The `NaN` issue can be solved by checking `Categorical.codes`

```python
In [7]: c = pd.Categorical([1, None, 2])

In [8]: c.codes < 0
Out[8]: array([False,  True, False])
```

The inf issue is deeper, and is present in Float64Index, so I think it should be split to its own PR.

```python
In [9]: c = pd.Categorical([1, np.inf])

In [10]: c.categories.astype(int)
Out[10]: Int64Index([1, -9223372036854775808], dtype='int64')
``` > That would be my preference. Did my earlier comment make sense?
> 
> The `NaN` issue can be solved by checking `Categorical.codes`
> 
> ```python
> In [7]: c = pd.Categorical([1, None, 2])
> 
> In [8]: c.codes < 0
> Out[8]: array([False,  True, False])
> ```
> 
> The inf issue is deeper, and is present in Float64Index, so I think it should be split to its own PR.
> 
> ```python
> In [9]: c = pd.Categorical([1, np.inf])
> 
> In [10]: c.categories.astype(int)
> Out[10]: Int64Index([1, -9223372036854775808], dtype='int64')
> ```

I believe so, you want to fix the `inf` casting behavior in one place (`Float64Index`) and then use the fixed `astype` method here?  Regarding the `nan` issue, is there a reason to prefer `self.codes < 0` over `self.isna()`? thanks @dsaxton 