CI: Run jobs on 2.3.x branch
Like https://github.com/pandas-dev/pandas/pull/56664 for 2.2.x
I suppose we could also directly merge it into the 2.3.x branch? But https://github.com/pandas-dev/pandas/pull/56664 first merged it into main and then backported it, so going to follow that same process here.

Merging, so it can actually be tested if it is working when backported