REF: Remove un-used attribute-pinning in plotting
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

xref #54485

@tacaswell can you confirm that matplotlib doesn't use any of these internally?  Removing these doesn't break any of our tests locally, but a lot of our plotting tests are basically smoke-tests.
I also believe that these are attributes that Pandas monkey-patches onto Matplotlib objects.

If you want to stash state related to unit conversion you can use the `Axis.units` attribute (best vias `Axis.set_units(...)`.

attn @ksunden Thanks @jbrockmendel  This commit appears to be causing an order of magnitude performance issue when plotting with a DatetimeIndex. See https://github.com/pandas-dev/pandas/issues/57035