CI: Fix pyarrow nightly build
EDIT: Looks like it was a stale cache issue. After deleting the cache (which the weekly job should help going forward), looks like pyarrow dev has an error
Tracking the change in double to temporal casting in https://github.com/apache/arrow/issues/36677 Tests seem to be passing and the casting workarounds seem to be the way forward with https://github.com/apache/arrow/issues/36677 so merging. Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

```
git checkout 2.0.x
git pull
```

2. Cherry pick the first parent branch of the this PR on top of the older branch:
```
git cherry-pick -x -m1 db27c36272fc3d7ce76cfecbeaa1eac704c2644a
```

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

```
git commit -am 'Backport PR #54110: CI: Fix pyarrow nightly build'
```

4. Push to a named branch:

```
git push YOURFORK 2.0.x:auto-backport-of-pr-54110-on-2.0.x
```

5. Create a PR against branch 2.0.x, I would have named this PR:

> "Backport PR #54110 on branch 2.0.x (CI: Fix pyarrow nightly build)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the `Still Needs Manual Backport` label once the PR gets merged.

If these instructions are inaccurate, feel free to [suggest an improvement](https://github.com/MeeseeksBox/MeeseeksDev).
                