Fix a few test failures on big-endian systems
These are all due to tests expecting little-endian dtypes, where in fact the endianness of the dtype is that of the host.

- [ ] closes #xxxx (Replace xxxx with the Github issue number) **No issue filed**
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature **Not applicable; fixes some tests on big-endian platforms**
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.
The force push was to remove a stray `from sys import byteorder` left over from when I rebased this from `v1.3.5` to `main`.