CI temporarily pin numpy
Looks like numba is failing with the latest numpy release https://github.com/numba/numba/issues/8615

Let's temporarily pin numpy to get CI green, and then lift the pin once it's solved in numba?
I guess you have to pin in the asv config as well? not strictly necessary for CI, as `--python=same` is used in that job - but yeah, probably better, thanks thx @MarcoGorelli  Could we remove numba in some of the envs instead? Because not testing with the latest numpy anywhere doesn't seem like a good situation? 
(I don't know the timeline of a fix fox this in numba, but the issue is already open a while) the 311-dev job tests with the latest (nightly) numpy, and that one doesn't have numba

perhaps let's wait a few days to see if there's any progress, and if not just remove numba from CI 'til they resolve it?