BUG: Avoid casting to double type unnecessarily when setting values i…
…n time delta column

- [x] closes #23511
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

When setting a timedelta column in a dataframe using `pd.Series` or array as index, there is some rounding error because the underlying TimeDeltaBlock is casted to double and back to int. This patch fixes the issue. 


Hello @icexelloss! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/core/internals/blocks.py`](https://github.com/icexelloss/pandas/blob/a485da25a9b20f606ed6d55c24fab78b523ee7d7/pandas/core/internals/blocks.py) !

 - There are no PEP8 issues in the file [`pandas/tests/indexing/test_timedelta.py`](https://github.com/icexelloss/pandas/blob/a485da25a9b20f606ed6d55c24fab78b523ee7d7/pandas/tests/indexing/test_timedelta.py) !

 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23462?src=pr&el=h1) Report
> Merging [#23462](https://codecov.io/gh/pandas-dev/pandas/pull/23462?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0bb24b7ac5c58bf9115485bba8c8b97442ab5059?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/23462/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/23462?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #23462   +/-   ##
=======================================
  Coverage   92.23%   92.23%           
=======================================
  Files         161      161           
  Lines       51197    51197           
=======================================
  Hits        47220    47220           
  Misses       3977     3977
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.61% <100%> (ø)` | :arrow_up: |
| #single | `42.27% <0%> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/23462?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/internals/blocks.py](https://codecov.io/gh/pandas-dev/pandas/pull/23462/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzL2Jsb2Nrcy5weQ==) | `94.02% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23462?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/23462?src=pr&el=footer). Last update [0bb24b7...62ab354](https://codecov.io/gh/pandas-dev/pandas/pull/23462?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 lgtm. ping on green. thanks @icexelloss  Thanks @jreback !