REGR: groupby.idxmin/idxmax wrong result on extreme values
- [x] closes #57040 (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

While working on this, I noticed that the `skipna` argument was also being ignored.

The only way I see solving this is adding a `seen` Boolean array instead of the sentinel values. cc @WillAyd
Thanks @rhshadrach 