TST: Fix broken test cases where Timedelta/Timestamp raise
Orthogonal to #28725, but will need to be rebased after that is merged.
see my comment on #28725  can you rebase rebased+green These are user facing changes right? Need a whatsnew? > These are user facing changes right? Need a whatsnew?

not unless you have an example of what is changed. these are broken behavior in an internal method, likely there is a lot of duplicated / unsused code (except by the tests). thanks @jbrockmendel 