CLN avoid more test warnings
Similar to https://github.com/pandas-dev/pandas/pull/50493

There are places when an implicit cast happens, but it's not the purpose of the test - the cast might as well be done explicitly then

This would help pave the way towards https://github.com/pandas-dev/pandas/pull/50424

---

I expect this to be the last precursor PR, after this one I'll raise a draft PR to try enforcing PDEP6 in some cases (note that merging would only take place _after_ 2.0 is released)
thx @MarcoGorelli 