BUG: to_csv casting datetimes in categorical to int
- [x] closes #40754
- [x] closes #18024
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them
- [x] whatsnew entry

i think the underlying problem is in `Categorical.astype(object)` Was not sure if this was intentional in the astype call. Will have a look again So I think I figured the issue out now.

``Categorical.astype`` was incorrect for object dtype as you suggested. But this does not directly help us in the ``to_csv`` case, becaause this has to respect a ``date_format`` in ``to_native_types``. We have to convert this to a DatetimeArray before doing the actual conversion.  thanks @phofl 