Backport PR #35898 on branch 1.1.x: CI: docker 32-bit linux build
Backport PR #35898
32 bit failures 
```
=========================== short test summary info ============================
FAILED pandas/tests/indexing/test_partial.py::TestPartialSetting::test_slice_irregular_datetime_index_with_nan
FAILED pandas/tests/window/test_grouper.py::TestGrouperGrouping::test_groupby_rolling_string_index
FAILED pandas/tests/window/test_grouper.py::TestGrouperGrouping::test_groupby_rolling_no_sort
= 3 failed, 69851 passed, 4429 skipped, 14962 deselected, 1047 xfailed, 6 xpassed, 12 warnings in 758.55s (0:12:38) =
```

This matches https://dev.azure.com/pandas-dev/pandas-wheels/_build/results?buildId=46211&view=logs&j=bb1c2637-64c6-57bd-9ea6-93823b2df951&t=9c62e7ff-391e-517d-ede1-af63bc6a25da with `test_groupby_nat_exclude` and `test_info_int_columns` fixed here.

will merge this and address these failures separately

`test_groupby_rolling_string_index` and `test_groupby_rolling_no_sort` only occuring on 1.1.

`test_slice_irregular_datetime_index_with_nan` is failing on master and only appeared on https://github.com/MacPython/pandas-wheels/pull/106 after the last commit so is recent and not picked up #35898