Made date time sources usage more explicit in setup.py
Not sure this fully solves #30236 but doesn't hurt and makes things more explicit
Hello @WillAyd! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:



There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers: 

##### Comment last updated at 2019-12-13 22:11:36 UTC hmm i am always leary of changing things like this, last time we did things didn't recompile properly locally. meaning if you then *just* change something that was formerly a source (but now only a depends) it didn't recompile unless you clean.

@jbrockmendel can you test out.

 I agree with both @WillAyd that this looks correct and is more explicit and @jreback that in the past this has been a fragile minefield.  On the margin I'm inclined to trust Will on this as it is easy to revert if problems pop up. Cool thanks both for the feedback. Just to clarify on this one I'm just removing sources that aren't used by the module at all, not necessarily converting from `sources` -> `depends`

The problem I'm trying to address is that with parallel builds everything in sources gets recompiled when specified as such, and I think race conditions could make things fail

This doesn't fully solve the recompilation problem but just removes the compilation step from modules that don't reference those C extensions at all, whether directly or through import of another module Any objection to rolling with this for now and reverting if issues pop up? Have it as a mini goal to get parallelized builds which I think this is a pre-cursor of +1 Cool I'll merge for now and obviously can revert if anything pops up

cc @pandas-dev/pandas-core for visibility