DOC: restore toctree maxdepth
Closes https://github.com/pandas-dev/pandas/issues/25131

I used a `maxdepth` of 3 now, instead of the original 4.

Now, the problem is that this makes the toctree on the main page (the one in the actual page body, not the one in the left sidebar) again very long and not very usable ..

cc @datapythonista @neili02
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=h1) Report
> Merging [#25134](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/d3c9d6e67491caf9046b0f8efd2d8d153b52b8e3?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25134/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #25134      +/-   ##
==========================================
+ Coverage   92.37%   92.37%   +<.01%     
==========================================
  Files         166      166              
  Lines       52408    52408              
==========================================
+ Hits        48411    48412       +1     
+ Misses       3997     3996       -1
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.79% <ø> (ø)` | :arrow_up: |
| #single | `42.86% <ø> (-0.01%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `88.13% <0%> (+0.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=footer). Last update [d3c9d6e...ade2608](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=h1) Report
> Merging [#25134](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/e28ae70ba8bd8b4174143543d1660421bc67887b?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/25134/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #25134   +/-   ##
=======================================
  Coverage   91.26%   91.26%           
=======================================
  Files         173      173           
  Lines       52968    52968           
=======================================
  Hits        48339    48339           
  Misses       4629     4629
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.83% <ø> (ø)` | :arrow_up: |
| #single | `41.71% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/config.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29uZmlnLnB5) | `87% <0%> (-0.05%)` | :arrow_down: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `83.53% <0%> (ø)` | :arrow_up: |
| [pandas/core/arrays/sparse.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL3NwYXJzZS5weQ==) | `92.17% <0%> (ø)` | :arrow_up: |
| [pandas/core/groupby/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9iYXNlLnB5) | `91.83% <0%> (ø)` | :arrow_up: |
| [pandas/core/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYmFzZS5weQ==) | `97.76% <0%> (ø)` | :arrow_up: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `96.79% <0%> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `93.65% <0%> (ø)` | :arrow_up: |
| [pandas/io/feather\_format.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2ZlYXRoZXJfZm9ybWF0LnB5) | `89.74% <0%> (ø)` | :arrow_up: |
| [pandas/core/groupby/groupby.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ3JvdXBieS9ncm91cGJ5LnB5) | `97.2% <0%> (ø)` | :arrow_up: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.57% <0%> (ø)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/pandas-dev/pandas/pull/25134/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=footer). Last update [e28ae70...789e7c7](https://codecov.io/gh/pandas-dev/pandas/pull/25134?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 -1 on this.

Didn't realise that the home page depth was affecting the sidebar levels (I understand that's this is what it's being fixed).

In `doc/source/conf.py`, in the `header` context we could add the `tocdepth`, which will be injected in every page. That's what it's supposed to control the sidebars.

Or I guess that creating an invisible toctree with depth 3 or 4 after this one should have the same effect (I guess the depth is being propagated from page to page after this toctree is being rendered). > Didn't realise that the home page depth was affecting the sidebar levels (I understand that's this is what it's being fixed).

Yes, that is what is being fixed. 
And IMO, the current *within page* navigation is quite broken. We have a lot of huge pages, and there is now no way to go to a specific section apart from scrolling.

>  we could add the `tocdepth`. That's what it's supposed to control the sidebars.

I am not sure that is what it is supposed to control. At least, from trying both in the pandas docs, as in a dummy test docs, it does not do that .. It's not really clear to me what it then actually *does* control. 
But the sidebar maxdepth seems to be solely determined by the main `toctree` entry on the main `index.rst`. All other combinations of tocdepths and maxdepths on the other pages / toctrees (that I tried, there are probably more ... :)) didn't influence the sidebar. 

> Or I guess that creating an invisible toctree with depth 3 or 4 after this one should have the same effect (I guess the depth is being propagated from page to page after this toctree is being rendered).

Yes, I tried that as well, but it cannot be combined with another *visible* toctree with less depth. Because it will simply put the two toctrees in the sidebar ...

---

It's a trade-off between a better side-bar navigation and a shorter (more useful) table of contents on the main index page. Personally, I find the side bar more important.

The only solution I see at the moment is to make the main toctree `:hidden:`, and create a shorter overview manually for the main index page (without using the `toctree` directive). Basically what eg xarray does: http://xarray.pydata.org/en/stable/ (their sidebar expands additional levels, but the listing on the body of the main page only has one level -> that one is constructed manually).

 
 BTW, I opened an issue about the :tocdepth: on sphinx: https://github.com/sphinx-doc/sphinx/issues/6033 @datapythonista I implemented what I describe above (last paragraph in https://github.com/pandas-dev/pandas/pull/25134#issuecomment-461359544):

- have the main index.rst toctree with a `:maxdepth:` of 3 instead of 2, but `:hidden:` -> it is used for the sidebar, but the long (with 3 levels) toctree is not shown on the front page main content
- manually insert a toctree-like list to mimic the toctree with a `:maxdepth:` of 2

I needed to edit out sphinx theme template to include hidden toctrees in the sidebar @jreback @TomAugspurger I would like to include this for 0.24.2. 

And before you complain about the added complexity (separate manual toctree to keep up to date): as far as I know, there is no way with sphinx to have both the sidebar toc go up to a level of 3, and have that same toctree shown on the main page with a depth up to a level of 2. See the discussion above. 
I would love to hear a better solution, but I already spent quite some time trying to figure this out, and currently my conclusion is that it is not possible with sphinx. @jorisvandenbossche thoughts on #25614 before this? I find the default TOC navigation that comes with that on the left hand side rather nice.  > thoughts on #25614 before this? I find the default TOC navigation that comes with that on the left hand side rather nice.

@WillAyd I am personally slighly -1 on hurrying that, but will try to answer there soon. But also, both PRs should be quite independent (except of the one-line change I made in the theme layout here, which would not be needed with the rtd theme. But the main part of this PR (changing the maxdepth etc) is also needed for the rtd theme) (failure is the sparse one which is fixed on master)

Going to merge this soon, any more comments? I think this is good to merge now (or soon. Maybe leave a couple hours).

This solves the biggest complain with the 0.24.0 reorganization. But it shouldn't preclude further improvements in navigation and restructuring of our very large pages. > This solves the biggest complain with the 0.24.0 reorganization. But it shouldn't preclude further improvements in navigation and restructuring of our very large pages.

Yes, for sure. This is only meant for patching up what we currently have to make it more usable again, not to be a final solution.