Backport PR #35582 on branch 1.1.x (BUG: to_timedelta fails on Int64 Series with null values)
Backport PR #35582: BUG: to_timedelta fails on Int64 Series with null values
known flaky test so merging

```
=========================== short test summary info ===========================
FAILED pandas/tests/io/parser/test_common.py::test_chunks_have_consistent_numerical_type[python]
= 1 failed, 72705 passed, 1743 skipped, 1054 xfailed, 58 warnings in 748.75s (0:12:28) =
```