BUG: Categorical.unique() preserves categories
 - [X] closes #13179
 - [x] tests added / passed
 - [X] passes ``git diff upstream/master | flake8 --diff``
 - [X] whatsnew entry

you are changing ``Categorical.unique`` which is a no-no. This should only be done in .groupby have a look here: https://github.com/pandas-dev/pandas/blob/master/pandas/core/groupby.py#L2314 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15439?src=pr&el=h1) Report
> Merging [#15439](https://codecov.io/gh/pandas-dev/pandas/pull/15439?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/f62e8f242d90d849e4854f4fe82e9ebb5b731d74?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.


```diff
@@            Coverage Diff             @@
##           master   #15439      +/-   ##
==========================================
+ Coverage   90.37%   90.37%   +<.01%     
==========================================
  Files         135      135              
  Lines       49476    49481       +5     
==========================================
+ Hits        44713    44720       +7     
+ Misses       4763     4761       -2
```


| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/15439?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/groupby.py](https://codecov.io/gh/pandas-dev/pandas/compare/f62e8f242d90d849e4854f4fe82e9ebb5b731d74...82ff3d4565e0ca41b4ccb47b61bdd3eb72b1ff7d?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS5weQ==) | `94.98% <100%> (+0.09%)` | :white_check_mark: |
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/compare/f62e8f242d90d849e4854f4fe82e9ebb5b731d74...82ff3d4565e0ca41b4ccb47b61bdd3eb72b1ff7d?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `96.87% <100%> (+0.03%)` | :white_check_mark: |
| [pandas/indexes/category.py](https://codecov.io/gh/pandas-dev/pandas/compare/f62e8f242d90d849e4854f4fe82e9ebb5b731d74...82ff3d4565e0ca41b4ccb47b61bdd3eb72b1ff7d?src=pr&el=tree#diff-cGFuZGFzL2luZGV4ZXMvY2F0ZWdvcnkucHk=) | `98.59% <100%> (ø)` | :white_check_mark: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15439?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/15439?src=pr&el=footer). Last update [f62e8f2...55733b8](https://codecov.io/gh/pandas-dev/pandas/compare/f62e8f242d90d849e4854f4fe82e9ebb5b731d74...55733b8c497b896662daf687969c80a2628beef6?el=footer&src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). > I only have one question: should we have different behaviour of sort=False depending on whether the categorical is ordered or not?
Because now the sort=False seems to be ignored in case of an ordered categorical?

hmm, @kernc can you take a look at the original issue (in the reference from the code) and see if you can divine what is happening? Indeed, the order is retained by `.unique()`, most recently edited in 29f1f42418d161b4690375acde5d5743bdd10772, _"BUG: Groupby(sort=False) with datetime-like Categorical raises ValueError"_ with several tests.

Semantically, I don't think the order should be lost if the categorical is ordered. The relationships `'low' < 'med' < 'high'` don't simply disappear if the data contains no `'med'`. Transitivity persists.
Remember that this whole debacle is due to [some Gimli desiring the groups be in random order](https://github.com/pandas-dev/pandas/issues/8868) when groupby sort=False, while in reality, few could care less. :laughing:

This lgtm; please edit to your liking. thanks @kernc nice PR!

keep em coming!