TST: Test for bug fixed during #22534 discussion
In the course of profiling #22534 a bug was found, but that PR was made redundant and the bug fixed in master.  This just adds a test for that bug.
Hello @jbrockmendel! Thanks for submitting the PR.

 - There are no PEP8 issues in the file [`pandas/tests/frame/test_arithmetic.py`](https://github.com/jbrockmendel/pandas/blob/f31407c7ad778d64e8b09a373b63513d6f45c7cf/pandas/tests/frame/test_arithmetic.py) !

