BUG: pandas-dev#58594
- [X] closes #58594 
- [X] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [X] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

No errors raised for all cases noted in https://github.com/pandas-dev/pandas/issues/58594.

```
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> empty_df = pd.DataFrame([])  # works
>>> df = pd.DataFrame([], index=empty_df.index)  # works
>>> df = pd.DataFrame([], index=[])  # works
>>> df = pd.DataFrame.from_records([])  # works
>>> df = pd.DataFrame.from_records([], index=empty_df.index)  # works now!
>>> df = pd.DataFrame.from_records([], index=[])  # works now!
``` Thanks @kirill-bash 