FIX: remove dependence on matplotlib.dates.epoch2num
- [ ] closes #34850
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

`converter._dt_to_float_ordinal(dt)` special cased `datetime64` objects, and used `matplotlib.dates.epoch2num`.  However, matplotlib has supported `datetime64` natively since Matplotlib 2.2 (https://github.com/matplotlib/matplotlib/pull/9779), so this special casing shouldn't be necessary (unless there is a subtlety I'm missing).  Matplotlib would also prefer to deprecate `epoch2num`, so this would help us streamline.  

I didn't check your support policy - if you need to support pre MPL 2.2, then I guess some try/except logic will need to go in here.  

ping @TomAugspurger @dstansby 
Thanks. I'm looking into this a bit too. Will take a closer look later. So, pandas has a quasi-public API at https://github.com/pandas-dev/pandas/blob/3e88e170a436b5e3cfdfc29f8a7416220658c616/pandas/plotting/_matplotlib/converter.py#L136 that gets tested in `test_converter.py` and assumes a matplotlib epoch of 0000-12-31.  Most plotting seems OK with the change in this PR, but the hard-coded tests that assume the epoch is the same as `datetime.toordinal()` naturally fail.  Not sure how you want to handle this dance - I think it would be easiest from the users point of view to just use the Matplotlib epoch and adjust the test (and the quasi-public results of calling `convert`).  But I don't claim expertise in how pandas users are used to handling datetime objects. Thanks for starting on this @jklymak. I've incorporated it into https://github.com/pandas-dev/pandas/pull/35393.