CI: Fail doc build on warning
(continuing a theme)

xref https://github.com/pandas-dev/pandas/issues/21564

We print the doc build output to a file and parse that file for errors. This is preferable to Sphinx'x built-in ability to raise on errors, since we will print warnings for *all* the errors present in a PR, rather than just the first (and immediately exiting the build).
Hello @TomAugspurger! Thanks for updating the PR.

 - There are no PEP8 issues in the file [`doc/make.py`](https://github.com/TomAugspurger/pandas/blob/378bd6dd7d664c64dee55ae15f2fd1198852a81b/doc/make.py) !

 - There are no PEP8 issues in the file [`pandas/core/generic.py`](https://github.com/TomAugspurger/pandas/blob/378bd6dd7d664c64dee55ae15f2fd1198852a81b/pandas/core/generic.py) !

 - There are no PEP8 issues in the file [`pandas/core/series.py`](https://github.com/TomAugspurger/pandas/blob/378bd6dd7d664c64dee55ae15f2fd1198852a81b/pandas/core/series.py) !

 - There are no PEP8 issues in the file [`pandas/core/window.py`](https://github.com/TomAugspurger/pandas/blob/378bd6dd7d664c64dee55ae15f2fd1198852a81b/pandas/core/window.py) !

 - There are no PEP8 issues in the file [`pandas/io/formats/style.py`](https://github.com/TomAugspurger/pandas/blob/378bd6dd7d664c64dee55ae15f2fd1198852a81b/pandas/io/formats/style.py) !



##### Comment last updated on September 20, 2018 at 20:51 Hours UTC # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=h1) Report
> Merging [#22743](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/79f7762c622c61e0dbf32575403206bcc9c402ab?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/22743/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master   #22743   +/-   ##
=======================================
  Coverage   92.17%   92.17%           
=======================================
  Files         169      169           
  Lines       50739    50739           
=======================================
  Hits        46767    46767           
  Misses       3972     3972
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.58% <ø> (ø)` | :arrow_up: |
| #single | `42.34% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/formats/style.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvc3R5bGUucHk=) | `96.43% <ø> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.67% <ø> (ø)` | :arrow_up: |
| [pandas/core/window.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvd2luZG93LnB5) | `96.27% <ø> (ø)` | :arrow_up: |
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `93.75% <ø> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=footer). Last update [79f7762...bbcd7bd](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 # [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=h1) Report
> Merging [#22743](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/0480f4c183a95712cb8ceaf5682c5b8dd02e0f21?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/22743/graphs/tree.svg?width=650&token=eZ4WkYLtcO&height=150&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #22743      +/-   ##
==========================================
+ Coverage   92.18%   92.18%   +<.01%     
==========================================
  Files         169      169              
  Lines       50804    50807       +3     
==========================================
+ Hits        46833    46836       +3     
  Misses       3971     3971
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `90.59% <ø> (ø)` | :arrow_up: |
| #single | `42.33% <ø> (ø)` | :arrow_up: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/core/series.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc2VyaWVzLnB5) | `93.74% <ø> (ø)` | :arrow_up: |
| [pandas/io/formats/style.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvc3R5bGUucHk=) | `96.43% <ø> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `96.67% <ø> (ø)` | :arrow_up: |
| [pandas/core/window.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvd2luZG93LnB5) | `96.28% <ø> (ø)` | :arrow_up: |
| [pandas/io/formats/excel.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2Zvcm1hdHMvZXhjZWwucHk=) | `97.4% <0%> (ø)` | :arrow_up: |
| [pandas/core/arrays/datetimelike.py](https://codecov.io/gh/pandas-dev/pandas/pull/22743/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvYXJyYXlzL2RhdGV0aW1lbGlrZS5weQ==) | `95.53% <0%> (+0.02%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=footer). Last update [0480f4c...b9331f1](https://codecov.io/gh/pandas-dev/pandas/pull/22743?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
 Pushed some changes to how things are checked

1. Not passing `-W` to sphinx-build. The fact that it errors immediately isn't great for CI. We would want a contributor to see all the mistakes made in there PR, not just the first.
2. Added a `--warnings-are-errors` flag to `make.py` to pass through `-W`, helpful for local debugging
3. In `build_docs.sh`, we redirect stdout and stderr to a log file.
4. In https://github.com/pandas-dev/pandas/pull/22743/files#diff-884b5131af7960906a35ecd1d041c7c6R343, we examine that log file for problems. Right now, it just looks for "WARNING:", but in the future I think that will just be a regular expression run against each line of the log.
5. If the linter finds warnings, we'll print them out and exit with 1, to fail travis

I've introduced an error in the whatsnew. We'll see if it fails. https://travis-ci.org/pandas-dev/pandas/jobs/431140070#L5314 found the test failure (and one more) but didn't fail the build.

I'll work on the linter reporting a bit now. The line-based one doesn't work for exceptions from ipython, since those are split across multiple lines. OK that was a fun little exercise. 

With the following log:

<details>

```
[autosummary] generating autosummary for: source/index.rst
[autosummary] writing to source/generated_single
Running Sphinx v1.7.5
Initializing Spelling Checker
loading pickled environment... done
[autosummary] generating autosummary for: 10min.rst, advanced.rst, api.rst, basics.rst, categorical.rst, comparison_with_r.rst, comparison_with_sas.rst, comparison_with_sql.rst, comparison_with_stata.rst, computation.rst, ..., r_interface.rst, release.rst, reshaping.rst, sparse.rst, text.rst, timedeltas.rst, timeseries.rst, tutorials.rst, visualization.rst, whatsnew.rst
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 2 source files that are out of date
updating environment: 0 added, 2 changed, 0 removed
reading sources... [ 50%] generated_single/pandas.DataFrame.head
source/whatsnew/v0.24.0.txt:191: WARNING: Bullet list ends without a blank line; unexpected unindent.

>>>-------------------------------------------------------------------------
Exception in /Users/taugspurger/sandbox/pandas/pandas/core/generic.py:docstring of pandas.DataFrame.head at block ending on line 11
Specify :okexcept: as an option in the ipython:: block to suppress this message
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-1-8b4ac6d3a3e1> in <module>()
----> 1 2 / 0

ZeroDivisionError: division by zero
<<<-------------------------------------------------------------------------

reading sources... [100%] index

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [ 50%] generated_single/pandas.DataFrame.head
writing output... [100%] index

generating indices... genindex py-modindex
writing additional pages... generated/pandas.core.common.isnull generated/pandas.core.common.notnull generated/pandas.core.reshape.get_dummies generated/pandas.tools.merge.concat generated/pandas.tools.merge.merge generated/pandas.tools.pivot.pivot_table generated/pandas.tseries.tools.to_datetime generated/pandas.io.clipboard.read_clipboard generated/pandas.io.excel.ExcelFile.parse generated/pandas.io.excel.read_excel generated/pandas.io.gbq.read_gbq generated/pandas.io.html.read_html generated/pandas.io.json.read_json generated/pandas.io.parsers.read_csv generated/pandas.io.parsers.read_fwf generated/pandas.io.parsers.read_table generated/pandas.io.pickle.read_pickle generated/pandas.io.pytables.HDFStore.append generated/pandas.io.pytables.HDFStore.get generated/pandas.io.pytables.HDFStore.put generated/pandas.io.pytables.HDFStore.select generated/pandas.io.pytables.read_hdf generated/pandas.io.sql.read_sql generated/pandas.io.sql.read_frame generated/pandas.io.sql.write_frame generated/pandas.io.stata.read_stata generated/pandas.tseries.resample.Resampler generated/pandas.tseries.resample.Resampler.__iter__ generated/pandas.tseries.resample.Resampler.agg generated/pandas.tseries.resample.Resampler.aggregate generated/pandas.tseries.resample.Resampler.apply generated/pandas.tseries.resample.Resampler.asfreq generated/pandas.tseries.resample.Resampler.ax generated/pandas.tseries.resample.Resampler.backfill generated/pandas.tseries.resample.Resampler.bfill generated/pandas.tseries.resample.Resampler.count generated/pandas.tseries.resample.Resampler.ffill generated/pandas.tseries.resample.Resampler.fillna generated/pandas.tseries.resample.Resampler.first generated/pandas.tseries.resample.Resampler.get_group generated/pandas.tseries.resample.Resampler.groups generated/pandas.tseries.resample.Resampler.indices generated/pandas.tseries.resample.Resampler.interpolate generated/pandas.tseries.resample.Resampler.last generated/pandas.tseries.resample.Resampler.max generated/pandas.tseries.resample.Resampler.mean generated/pandas.tseries.resample.Resampler.median generated/pandas.tseries.resample.Resampler.min generated/pandas.tseries.resample.Resampler.ndim generated/pandas.tseries.resample.Resampler.nearest generated/pandas.tseries.resample.Resampler.ngroups generated/pandas.tseries.resample.Resampler.nunique generated/pandas.tseries.resample.Resampler.obj generated/pandas.tseries.resample.Resampler.ohlc generated/pandas.tseries.resample.Resampler.pad generated/pandas.tseries.resample.Resampler.pipe generated/pandas.tseries.resample.Resampler.plot generated/pandas.tseries.resample.Resampler.prod generated/pandas.tseries.resample.Resampler.quantile generated/pandas.tseries.resample.Resampler.sem generated/pandas.tseries.resample.Resampler.size generated/pandas.tseries.resample.Resampler.std generated/pandas.tseries.resample.Resampler.sum generated/pandas.tseries.resample.Resampler.transform generated/pandas.tseries.resample.Resampler.var generated/pandas.formats.style.Styler generated/pandas.formats.style.Styler.apply generated/pandas.formats.style.Styler.applymap generated/pandas.formats.style.Styler.background_gradient generated/pandas.formats.style.Styler.bar generated/pandas.formats.style.Styler.clear generated/pandas.formats.style.Styler.env generated/pandas.formats.style.Styler.export generated/pandas.formats.style.Styler.format generated/pandas.formats.style.Styler.from_custom_template generated/pandas.formats.style.Styler.hide_columns generated/pandas.formats.style.Styler.hide_index generated/pandas.formats.style.Styler.highlight_max generated/pandas.formats.style.Styler.highlight_min generated/pandas.formats.style.Styler.highlight_null generated/pandas.formats.style.Styler.loader generated/pandas.formats.style.Styler.render generated/pandas.format.style.Styler.set_caption generated/pandas.formats.style.Styler.set_precision generated/pandas.formats.style.Styler.set_properties generated/pandas.formats.style.Styler.set_table_attributes generated/pandas.formats.style.Styler.set_table_styles generated/pandas.formats.style.Styler.set_uuid generated/pandas.formats.style.Styler.template generated/pandas.formats.style.Styler.to_excel generated/pandas.formats.style.Styler.use generated/pandas.formats.style.Styler.where search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.

The HTML pages are in build/html.
```

</details>

The linter outputs

```
2 failures

Failure [0]: source/whatsnew/v0.24.0.txt:191: WARNING: Bullet list ends without a blank line; unexpected unindent.
Failure [1]: >>>-------------------------------------------------------------------------
Exception in /Users/taugspurger/sandbox/pandas/pandas/core/generic.py:docstring of pandas.DataFrame.head at block ending on line 11
Specify :okexcept: as an option in the ipython:: block to suppress this message
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-1-8b4ac6d3a3e1> in <module>()
----> 1 2 / 0

ZeroDivisionError: division by zero
<<<-------------------------------------------------------------------------
```

Can add more rules as needed. LMK if people don't like this kind of parser. https://travis-ci.org/pandas-dev/pandas/jobs/431185566#L5353 

I think it's still not failing because Travis is calling `run_build_docs.sh` which in turn calls `build_docs` in a subprocess and always exits with 0. I'm going to restructure things a bit. We'll need to either figure out the rpy2 on travis issue, or figure out a way to "whitelist" errors. Maybe this would be a good reason to move the doc build to azure. I'm going to split off the changes to the docs to fix the warnings. We can followup later with the infrastructure to fail the build. That sounds good. Did you see that `sphinx-build` has an argument `-W` that should fail the doc build if warnings are found? I'm working on a PR to speed up running `doc/make.py` with `--single`, will check also if that option works. The problem with `-W` is that it fails the build immediately and stops
building the docs. We would rather have the doc build finish to completion,
but fail CI if any warnings were present.

On Sun, Dec 16, 2018 at 8:46 PM Marc Garcia <notifications@github.com>
wrote:

> That sounds good. Did you see that sphinx-build has an argument -W that
> should fail the doc build if warnings are found? I'm working on a PR to
> speed up running doc/make.py with --single, will check also if that
> option works.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/22743#issuecomment-447706320>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIhQZmAHxBaoXDVlgF9BzQgdqHk1Aks5u5wV5gaJpZM4WuRqM>
> .
>
 Will you continue the work to split off the changes in this PR, or will you open new ones and should we close this?

I'm not sure if there are many warnings left of the ones you were fixing here. I think many have been fixed when linting the code in the docs, and I've been fixing some too.

I agree that ideally we'd like to run the whole doc build and not cancel at the first warning. But given that cancelling is implemented just adding a parameter to the build call, and the complexity needed to run the whole build, I'd prefer to just use `-W`. In the PRs, we are discarding the generated docs anyway, and if that makes the build fail, we shouldn't have any warnings in master. So, in practice, the difference is that we'll just see the first warning in the CI and not all them. Which I don't think justifies the extra complexity in the build. I think that most of the warnings are fixed on master.

I don't think I would support adding a `-W` flag to the build. That was my
first approach, and even locally the feedback loop was just too slow. Doing
it on the CI would be even worse.
I'd prefer the status quo of manually checking for warnings over failing on
the first warning.

Perhaps we could push this upstream to sphinx though.

On Sun, Dec 30, 2018 at 6:06 PM Marc Garcia <notifications@github.com>
wrote:

> Will you continue the work to split off the changes in this PR, or will
> you open new ones and should we close this?
>
> I'm not sure if there are many warnings left of the ones you were fixing
> here. I think many have been fixed when linting the code in the docs, and
> I've been fixing some too.
>
> I agree that ideally we'd like to run the whole doc build and not cancel
> at the first warning. But given that cancelling is implemented just adding
> a parameter to the build call, and the complexity needed to run the whole
> build, I'd prefer to just use -W. In the PRs, we are discarding the
> generated docs anyway, and if that makes the build fail, we shouldn't have
> any warnings in master. So, in practice, the difference is that we'll just
> see the first warning in the CI and not all them. Which I don't think
> justifies the extra complexity in the build.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/22743#issuecomment-450595898>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIqK9qsBesAU89lH0q6-9Ut-WD6Mmks5u-VULgaJpZM4WuRqM>
> .
>
 Ahh, maybe it's already done? Does the `--keep-going` flag do what we need?


```
bash-4.4$ make html SPHINXOPTS='-W --keep-going'
Running Sphinx v1.8.3
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
/private/tmp/sphinx-tst/index.rst:14: WARNING: undefined label: a (if the
link has no caption the label must precede a section header)
/private/tmp/sphinx-tst/index.rst:16: WARNING: undefined label: b (if the
link has no caption the label must precede a section header)
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build finished with problems, 2 warnings.
make: *** [html] Error 1
```

That would be great.

On Mon, Dec 31, 2018 at 7:38 AM Tom Augspurger <tom.augspurger88@gmail.com>
wrote:

> I think that most of the warnings are fixed on master.
>
> I don't think I would support adding a `-W` flag to the build. That was my
> first approach, and even locally the feedback loop was just too slow. Doing
> it on the CI would be even worse.
> I'd prefer the status quo of manually checking for warnings over failing
> on the first warning.
>
> Perhaps we could push this upstream to sphinx though.
>
> On Sun, Dec 30, 2018 at 6:06 PM Marc Garcia <notifications@github.com>
> wrote:
>
>> Will you continue the work to split off the changes in this PR, or will
>> you open new ones and should we close this?
>>
>> I'm not sure if there are many warnings left of the ones you were fixing
>> here. I think many have been fixed when linting the code in the docs, and
>> I've been fixing some too.
>>
>> I agree that ideally we'd like to run the whole doc build and not cancel
>> at the first warning. But given that cancelling is implemented just adding
>> a parameter to the build call, and the complexity needed to run the whole
>> build, I'd prefer to just use -W. In the PRs, we are discarding the
>> generated docs anyway, and if that makes the build fail, we shouldn't have
>> any warnings in master. So, in practice, the difference is that we'll just
>> see the first warning in the CI and not all them. Which I don't think
>> justifies the extra complexity in the build.
>>
>> —
>> You are receiving this because you were mentioned.
>> Reply to this email directly, view it on GitHub
>> <https://github.com/pandas-dev/pandas/pull/22743#issuecomment-450595898>,
>> or mute the thread
>> <https://github.com/notifications/unsubscribe-auth/ABQHIqK9qsBesAU89lH0q6-9Ut-WD6Mmks5u-VULgaJpZM4WuRqM>
>> .
>>
>
 Yes, didn't see the option, but it works as expected. :)

I had to make couple of changes to make our script behave properly when setting this option, but implemented it in #24523.

Do you want to take a look at this PR and see if there is anything you want to commit. Otherwise we can close it.  Closing this, as failing the doc build has been implemented in #24523, and I think all the warnings fixed here have already been fixed somewhere else. Feel free to reopen if there is anything that could be merged from here, and you want to reuse this PR for it. I am not sure #24523 is doing everything that this PR was doing? 

Didn't check in detail, but eg here Tom added some regexes for catching ipython directive errors. Are those handled as well in #24523? (or, Tom, what was the reason you needed to do that here?) I don't see those regex. I implemented a regex to find wrong spaces in the directives in #24650.

I couldn't find anything in this PR that is not fixed already (at least the parts generating warnings). But as said, feel free to reopen if it's still useful. > I don't see those regex.

The `lint_log` function here: https://github.com/pandas-dev/pandas/pull/22743/files#diff-884b5131af7960906a35ecd1d041c7c6R347 oh, I see. I thought you meant a regex over the rst files, not the log. I assume sphinx will return with an exit status != 0 if the process raises those (with the `-W` option). So, I don't think those are useful anymore. > I assume sphinx will return with an exit status != 0 if the process raises those (with the -W option).

I am not sure that is the case. From a quick test, sphinx doesn't seem to regard an IPython warning as a proper sphinx warning (but that might something that could be fixed in IPython). 

(Although that also relates to our use of `ipython_warning_is_error = False`, so erroring directly is possible (but we choose not to do that, as it stops the build), but letting sphinx turn a warning into an error, not sure about that) > but letting sphinx turn a warning into an error,

I think you meant error into warning (and then into an error)? IIUC our
ideal workflow is

1. We set a new IPython setting `ipython_error_is_warning = True` (maybe
exception_is_warning)
2. An error in an IPython block emits a warning, which sphinx logs, and the
doc build continues
3. Sphinx finishes, sees that there was an IPytyhon warning, and fails the
build



On Mon, Jan 7, 2019 at 6:51 AM Joris Van den Bossche <
notifications@github.com> wrote:

> I assume sphinx will return with an exit status != 0 if the process raises
> those (with the -W option).
>
> I am not sure that is the case. From a quick test, sphinx doesn't seem to
> regard an IPython warning as a proper sphinx warning (but that might
> something that could be fixed in IPython).
>
> (Although that also relates to our use of ipython_warning_is_error = False,
> so erroring directly is possible (but we choose not to do that, as it stops
> the build), but letting sphinx turn a warning into an error, not sure about
> that)
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/pandas-dev/pandas/pull/22743#issuecomment-451925195>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ABQHIqDSnFZa3UCPp3hlWPd7_9v_XgLLks5vA0LngaJpZM4WuRqM>
> .
>
 > I think you meant error into warning (and then into an error)?

Well, whether IPython directly emits a warning, or we have a way to turn their error in a warning, that doesn't matter too much to me :-) As long as the end result is that there is a warning, that can be turned into an exit code error with sphinx.

I would say that ideally, IPython uses the proper sphinx warning system and issues a sphinx warning instead of printing their warning to stdout, so that the sphinx warning machinery works with them properly? Yep. I'll raise an issue upstream.