API: dont do inference on object-dtype arithmetic results
- [ ] closes #xxxx (Replace xxxx with the GitHub issue number)
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [x] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Broken off from #49714, which also changes TimedeltaArray inference behavior.  This is exclusively Index/Series/DataFrame.
Thanks @jbrockmendel  Looks like this caused a slowdown. Is this expected?

https://asv-runner.github.io/asv-collection/pandas/#arithmetic.NumericInferOps.time_add?p-dtype=%3Cclass%20'numpy.int8'%3E best guess is #50306 will address that Sounds reasonable, thanks for the quick response