BUG to_timedelta was raising ValueError w pd.NA
- [x] closes ##52909
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

I think this is partially fixing #40566 - the first 3 cases on the issue description are no longer failing.

Edit: 
Now I can't see which test fails.

 Thanks @MarcoGorelli for reviewing & merging. I even got a cat! :-) Thanks @DeaMariaLeon  Thank you for merging @mroeschke  i think this fixed #38509 too!