FIX: Reject any non-array, non-ns datetimes and timedeltas
Done at euroSciPy2022! Fixes #48312.

On suggestion of @jorisvandenbossche, we opted to raise an error instead of converting to `[ns]` in order to make it easier in the future to add support for other granularity levels.

co-author: @kislovskiy

- [x] closes #48312
- [x] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [x] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [x] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

Hello @brisvag! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:





There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2022-09-05 09:54:15 UTC This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this. I don't know how to solve the build error without help, unfortunately, so I would appreciate some suggestions. Thanks for the pull request, but it appears to have gone stale. Additionally, it appears this behavior may head in a different direction (https://github.com/pandas-dev/pandas/issues/48312#issuecomment-1278345347 - happy to have your input!) Closing until the discussion is resolved there 