BUG: disallow invalid dtype to CategoricalDtype._from_values_or_dtype
- [ ] closes #xxxx
- [x] tests added / passed
- [x] passes `black pandas`
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

xref #32167.  Together with that, this will allow us to simplify the CategoricalIndex constructor.
#32167 first?  > #32167 first?

Any order should be OK