CI: Add regular 3.11 pipeline
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

cc @MarcoGorelli 

Looks like using mamba for the env fixes the installation issue, but we have some weird segfault on the 3.11 single cpu build `--extra-index-url` instead of `-i` seems to have fixed it too https://github.com/pandas-dev/pandas/pull/50698

if that works, would this one still be necessary? This one takes precedence over the dev build. It's kinda unrelated but after seeing the failures today I remembered that I wanted to look into it anyway.

The dev build only tests with our required dependencies while this one enables a full test-run on 3.11 (except numba since it's not compatible with 3.11 yet). This would give us the same test coverage as the other supported python versions.

Have to get the ci passing before that though. pytables is not yet available for 3.11 on conda-forge. Last build was green on 3.11, merging main to get up to date. @lithomas1 would you mind having a look? Merging to get our stuff tested properly on 3.11