TST/CI: Fix numexpr test failing in downstream build
- [x] tests added / passed
- [x] Ensure all linting tests pass, see [here](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit) for how to run them


@meeseeksdev backport 1.4.x Something went wrong ... Please have a look at my logs.