Remove old warnings (plus some useless code)
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

This removes some warnings which @jreback [suggested](https://github.com/pandas-dev/pandas/pull/17934#discussion_r146225334) to split from #17934 , plus some other useless, probably obsolete, lines of code.
I guess we never put these deprecations in https://github.com/pandas-dev/pandas/issues/6581
 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@a355ed2`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18022/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##             master   #18022   +/-   ##
=========================================
  Coverage          ?   91.22%           
=========================================
  Files             ?      163           
  Lines             ?    50089           
  Branches          ?        0           
=========================================
  Hits              ?    45693           
  Misses            ?     4396           
  Partials          ?        0
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.03% <100%> (?)` | |
| #single | `40.25% <50%> (?)` | |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `95.74% <100%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=footer). Last update [a355ed2...754ced2](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@a355ed2`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18022/graphs/tree.svg?width=650&src=pr&token=eZ4WkYLtcO&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##             master   #18022   +/-   ##
=========================================
  Coverage          ?   91.22%           
=========================================
  Files             ?      163           
  Lines             ?    50089           
  Branches          ?        0           
=========================================
  Hits              ?    45693           
  Misses            ?     4396           
  Partials          ?        0
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.03% <100%> (?)` | |
| #single | `40.25% <50%> (?)` | |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `95.74% <100%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=footer). Last update [a355ed2...754ced2](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@a355ed2`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18022/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##             master   #18022   +/-   ##
=========================================
  Coverage          ?   91.22%           
=========================================
  Files             ?      163           
  Lines             ?    50089           
  Branches          ?        0           
=========================================
  Hits              ?    45693           
  Misses            ?     4396           
  Partials          ?        0
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.03% <100%> (?)` | |
| #single | `40.25% <50%> (?)` | |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `95.74% <100%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=footer). Last update [a355ed2...754ced2](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=h1) Report
> Merging [#18022](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/96a527434a6138b291c1b4a782bd2793cff51f74?src=pr&el=desc) will **decrease** coverage by `0.04%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18022/graphs/tree.svg?token=eZ4WkYLtcO&width=650&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18022      +/-   ##
==========================================
- Coverage   91.43%   91.39%   -0.05%     
==========================================
  Files         163      163              
  Lines       50091    50083       -8     
==========================================
- Hits        45800    45771      -29     
- Misses       4291     4312      +21
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.19% <100%> (-0.03%)` | :arrow_down: |
| #single | `40.36% <37.5%> (-0.06%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/categorical.py](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY2F0ZWdvcmljYWwucHk=) | `95.75% <100%> (-0.05%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `63.38% <0%> (-1.82%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.8% <0%> (-0.1%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=footer). Last update [96a5274...56656e7](https://codecov.io/gh/pandas-dev/pandas/pull/18022?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 I tried to remove the call to ``maybe_infer_to_datetimelike``, but [some tests break](https://travis-ci.org/pandas-dev/pandas/builds/300411039). In another PR I can see if we can fix this inside ``_sanitize_array``. as far as https://github.com/pandas-dev/pandas/pull/18022#issuecomment-343647244
e.g. trying to remove some sanitizing code, would take that as a followup @jreback ping thanks @toobaz nice PR!