<p dir="auto">The PCG generator used by Numpy has a significant amount self-correlation. That is, for each sequence generated from a seed there is a large number of correlated, nonoverlapping sequences starting from other seeds. By "correlated" I mean that interleaving two such sequences and testing the result you obtain failures that did not appear in each sequence individually.</p>
<p dir="auto">The probability that two generators out of large set of terminals get two of those sequences is nonnegligible. Why this happens from a mathematical viewpoint is well known but it is explained here in detail: <a href="http://prng.di.unimi.it/pcg.pgp" rel="nofollow">http://prng.di.unimi.it/pcg.pgp</a> (see "Subsequences within the same generator").</p>
<p dir="auto">To show this problem directly, I wrote this simple C program reusing the Numpy code: <a href="http://prng.di.unimi.it/intpcgnumpy.c" rel="nofollow">http://prng.di.unimi.it/intpcgnumpy.c</a> . The program takes two 128-bit states of two generators (with the same LCG constant or "stream") in the form of high and low bits, interleaves their output and writes it in binary form. Once we send it through PractRand, we should see no statistical failure, as the two streams should be independent. But if try to start from two states with the same 64 lower bits, you get:</p>
<p dir="auto">./intpcgnumpy 0x596d84dfefec2fc7 0x6b79f81ab9f3e37b 0x8d7deae980a64ab0 0x6b79f81ab9f3e37b | stdbuf -oL ~/svn/c/xorshift/practrand/RNG_test stdin -tf 2 -te 1 -tlmaxonly -multithreaded<br>
RNG_test using PractRand version 0.94<br>
RNG = RNG_stdin, seed = unknown<br>
test set = expanded, folding = extra</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="rng=RNG_stdin, seed=unknown
length= 128 megabytes (2^27 bytes), time= 2.2 seconds
  Test Name                         Raw       Processed     Evaluation
  BCFN(0+0,13-2,T)                  R= +27.6  p =  1.0e-13    FAIL
  BCFN(0+1,13-2,T)                  R= +68.0  p =  2.3e-34    FAIL !!!
  BCFN(0+2,13-3,T)                  R= +90.8  p =  8.8e-43    FAIL !!!
  BCFN(0+3,13-3,T)                  R=+120.6  p =  6.9e-57    FAIL !!!!
  DC6-6x2Bytes-1                    R=  +8.9  p =  4.0e-5   mildly suspicious
  DC6-5x4Bytes-1                    R= +15.7  p =  4.3e-9   very suspicious
  [Low1/8]BCFN(0+0,13-4,T)          R= +11.6  p =  4.9e-5   unusual
  ...and 1074 test result(s) without anomalies"><pre class="notranslate"><code class="notranslate">rng=RNG_stdin, seed=unknown
length= 128 megabytes (2^27 bytes), time= 2.2 seconds
  Test Name                         Raw       Processed     Evaluation
  BCFN(0+0,13-2,T)                  R= +27.6  p =  1.0e-13    FAIL
  BCFN(0+1,13-2,T)                  R= +68.0  p =  2.3e-34    FAIL !!!
  BCFN(0+2,13-3,T)                  R= +90.8  p =  8.8e-43    FAIL !!!
  BCFN(0+3,13-3,T)                  R=+120.6  p =  6.9e-57    FAIL !!!!
  DC6-6x2Bytes-1                    R=  +8.9  p =  4.0e-5   mildly suspicious
  DC6-5x4Bytes-1                    R= +15.7  p =  4.3e-9   very suspicious
  [Low1/8]BCFN(0+0,13-4,T)          R= +11.6  p =  4.9e-5   unusual
  ...and 1074 test result(s) without anomalies
</code></pre></div>
<p dir="auto">You can even go lower—you just need the same 58 lower bits:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="./intpcgnumpy 0x596d84dfefec2fc7 0x0579f81ab9f3e37b 0x8d7deae980a64ab0 0x6b79f81ab9f3e37b | stdbuf -oL ~/svn/c/xorshift/practrand/RNG_test stdin -tf 2 -te 1 -tlmaxonly -multithreaded

[...]
rng=RNG_stdin, seed=unknown
length= 32 gigabytes (2^35 bytes), time= 453 seconds
  Test Name                         Raw       Processed     Evaluation
  [Low1/16]FPF-14+6/32:cross        R= +11.6  p =  4.0e-10   VERY SUSPICIOUS
  [Low1/32]FPF-14+6/32:cross        R= +16.5  p =  3.2e-14    FAIL
  [Low1/32]FPF-14+6/16:cross        R= +12.8  p =  3.8e-11   VERY SUSPICIOUS
  [Low1/64]FPF-14+6/64:cross        R=  +6.8  p =  4.8e-6   mildly suspicious
  [Low1/64]FPF-14+6/32:cross        R=  +6.0  p =  1.9e-5   unusual
  [Low1/64]FPF-14+6/16:cross        R=  +5.5  p =  5.8e-5   unusual
  [Low4/32]FPF-14+6/64:all          R=  +5.8  p =  5.9e-5   unusual
  [Low4/32]FPF-14+6/32:(0,14-0)     R=  +7.7  p =  1.0e-6   unusual
  [Low4/32]FPF-14+6/32:(1,14-0)     R=  +7.7  p =  9.1e-7   unusual
  [Low4/32]FPF-14+6/32:all          R=  +6.5  p =  1.3e-5   unusual
  [Low4/64]FPF-14+6/64:all          R=  +5.9  p =  5.1e-5   unusual
  [Low4/64]FPF-14+6/64:cross        R=  +8.2  p =  3.0e-7   suspicious
  [Low4/64]FPF-14+6/32:(0,14-0)     R=  +7.6  p =  1.0e-6   unusual
  [Low8/64]FPF-14+6/64:(0,14-0)     R= +17.0  p =  2.2e-15    FAIL
  [Low8/64]FPF-14+6/64:(1,14-0)     R=  +9.1  p =  5.1e-8   mildly suspicious
  [Low8/64]FPF-14+6/64:all          R= +12.7  p =  2.1e-11   VERY SUSPICIOUS
  [Low8/64]FPF-14+6/32:(0,14-0)     R= +12.8  p =  1.7e-11   VERY SUSPICIOUS
  [Low8/64]FPF-14+6/32:all          R= +11.0  p =  9.3e-10   VERY SUSPICIOUS
  ...and 1696 test result(s) without anomalies"><pre class="notranslate"><code class="notranslate">./intpcgnumpy 0x596d84dfefec2fc7 0x0579f81ab9f3e37b 0x8d7deae980a64ab0 0x6b79f81ab9f3e37b | stdbuf -oL ~/svn/c/xorshift/practrand/RNG_test stdin -tf 2 -te 1 -tlmaxonly -multithreaded

[...]
rng=RNG_stdin, seed=unknown
length= 32 gigabytes (2^35 bytes), time= 453 seconds
  Test Name                         Raw       Processed     Evaluation
  [Low1/16]FPF-14+6/32:cross        R= +11.6  p =  4.0e-10   VERY SUSPICIOUS
  [Low1/32]FPF-14+6/32:cross        R= +16.5  p =  3.2e-14    FAIL
  [Low1/32]FPF-14+6/16:cross        R= +12.8  p =  3.8e-11   VERY SUSPICIOUS
  [Low1/64]FPF-14+6/64:cross        R=  +6.8  p =  4.8e-6   mildly suspicious
  [Low1/64]FPF-14+6/32:cross        R=  +6.0  p =  1.9e-5   unusual
  [Low1/64]FPF-14+6/16:cross        R=  +5.5  p =  5.8e-5   unusual
  [Low4/32]FPF-14+6/64:all          R=  +5.8  p =  5.9e-5   unusual
  [Low4/32]FPF-14+6/32:(0,14-0)     R=  +7.7  p =  1.0e-6   unusual
  [Low4/32]FPF-14+6/32:(1,14-0)     R=  +7.7  p =  9.1e-7   unusual
  [Low4/32]FPF-14+6/32:all          R=  +6.5  p =  1.3e-5   unusual
  [Low4/64]FPF-14+6/64:all          R=  +5.9  p =  5.1e-5   unusual
  [Low4/64]FPF-14+6/64:cross        R=  +8.2  p =  3.0e-7   suspicious
  [Low4/64]FPF-14+6/32:(0,14-0)     R=  +7.6  p =  1.0e-6   unusual
  [Low8/64]FPF-14+6/64:(0,14-0)     R= +17.0  p =  2.2e-15    FAIL
  [Low8/64]FPF-14+6/64:(1,14-0)     R=  +9.1  p =  5.1e-8   mildly suspicious
  [Low8/64]FPF-14+6/64:all          R= +12.7  p =  2.1e-11   VERY SUSPICIOUS
  [Low8/64]FPF-14+6/32:(0,14-0)     R= +12.8  p =  1.7e-11   VERY SUSPICIOUS
  [Low8/64]FPF-14+6/32:all          R= +11.0  p =  9.3e-10   VERY SUSPICIOUS
  ...and 1696 test result(s) without anomalies
</code></pre></div>
<p dir="auto">Note that to get more the 50% probability that two generators start from two correlated seed (chosen at random) you need just about half a million generators starting at random (birthday paradox). And if you consider the probability that they do not exactly start from the same state, but have significant overlapping correlating sequences, you need much less.</p>
<p dir="auto">Any sensible generator from the literature will <em>not</em> behave like that. You can choose adversarially any two starting states of MRG32k3a, SFC64, CMWC, xoshiro256++, etc., and as long as you generate nonoverlapping sequences you will not see the failures above. This is a major drawback that can pop up when a number of devices uses the generator and one assumes (as it should be) that pairwise those sequences should not show correlation. The correlation can induce unwanted behavior that is hard to detect.</p>
<p dir="auto">Please at least document somewhere that the generator should not be used on multiple terminals or in a highly parallel environment.</p>
<p dir="auto">The same can happen with different "streams", as the sequences generated by an LCG by changing the additive constant are all the same modulo a change of sign and an additive constant. You can see some discussion here: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="522561054" data-permission-text="Title is private" data-url="https://github.com/rust-random/rand/issues/907" data-hovercard-type="issue" data-hovercard-url="/rust-random/rand/issues/907/hovercard" href="https://github.com/rust-random/rand/issues/907">rust-random/rand#907</a> and a full mathematical discussion of the problem here: <a href="https://arxiv.org/abs/2001.05304" rel="nofollow">https://arxiv.org/abs/2001.05304</a> .</p>
<p dir="auto">I wanted to build the PDF documentation (using Sphinx 2.2.1 and current versions of scipy, matplotlib, etc from <code class="notranslate">pip install</code> and using numpy from git at <a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/numpy/numpy/commit/b83f10ef7ee766bf30ccfa563b6cc8f7fd38a4c8/hovercard" href="https://github.com/numpy/numpy/commit/b83f10ef7ee766bf30ccfa563b6cc8f7fd38a4c8"><tt>b83f10e</tt></a>) and was disconcerted that it actually succeeded <g-emoji class="g-emoji" alias="wink" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f609.png">😉</g-emoji></p>
<p dir="auto">Turns out after checking that <a href="https://github.com/numpy/numpy/blob/master/doc/source/conf.py">https://github.com/numpy/numpy/blob/master/doc/source/conf.py</a> uses removed <code class="notranslate">latex_preamble</code> variable (it was removed at Sphinx 1.6.1). Correct syntax is now with <code class="notranslate">latex_elements['preamble']</code>. I did that, and obtained the failure on building <code class="notranslate">numpy-ref.pdf</code> that I was expecting:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="! Extra }, or forgotten \endgroup.
\@endpbox ...\@arstrutbox \color@endgroup \egroup 
                                                  \the \LT@p@ftn \global \LT...
l.131934 \item[{Parameters}]
                             \leavevmode"><pre class="notranslate"><code class="notranslate">! Extra }, or forgotten \endgroup.
\@endpbox ...\@arstrutbox \color@endgroup \egroup 
                                                  \the \LT@p@ftn \global \LT...
l.131934 \item[{Parameters}]
                             \leavevmode
</code></pre></div>
<p dir="auto">This is caused by these lines</p>
<p dir="auto"></p><div class="Box Box--condensed my-2">
  <div class="Box-header f6">
    <p class="mb-0 text-bold">
      <a href="https://github.com/numpy/numpy/blob/b83f10ef7ee766bf30ccfa563b6cc8f7fd38a4c8/doc/source/conf.py#L187-L189">numpy/doc/source/conf.py</a>
    </p>
    <p class="mb-0 color-fg-muted">
        Lines 187 to 189
      in
      <a data-pjax="true" class="commit-tease-sha" href="/numpy/numpy/commit/b83f10ef7ee766bf30ccfa563b6cc8f7fd38a4c8">b83f10e</a>
    </p>
  </div>
  <div itemprop="text" class="Box-body p-0 blob-wrapper blob-wrapper-embedded data">
    <table class="highlight tab-size mb-0 js-file-line-container" data-tab-size="8" data-paste-markdown-skip="">

        <tbody><tr class="border-0">
          <td id="L187" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="187"></td>
          <td id="LC187" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-s">\usepackage{expdlist}</span> </td>
        </tr>

        <tr class="border-0">
          <td id="L188" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="188"></td>
          <td id="LC188" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-s">\let\latexdescription=\description</span> </td>
        </tr>

        <tr class="border-0">
          <td id="L189" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="189"></td>
          <td id="LC189" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-s">\def\description{\latexdescription{}{} \breaklabel}</span> </td>
        </tr>
    </tbody></table>
  </div>
</div>
<p></p>
<p dir="auto">I know two fixes:</p>
<ul dir="auto">
<li>add <code class="notranslate">\usepackage{enumitem}</code>. I don't recommend that because the fact that it fixes the <code class="notranslate">expdlist</code> breakage appears unintentional, it is sheer luck (and I tested it on only one small example, not the full <code class="notranslate">numpy-ref.tex</code>). In the past Sphinx users used <code class="notranslate">enumitem</code> to avoid <code class="notranslate">Too deeply nested</code> error, but since Sphinx 1.5, there is <code class="notranslate">latex_elements['maxlistdepth']</code> to fix that LaTeX limitation:</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="latex_elements['maxlistdepth'] = '10'  # should be enough, increase at will (don't use '2048' as this would create needlessly thousands of TeX macros)"><pre class="notranslate"><code class="notranslate">latex_elements['maxlistdepth'] = '10'  # should be enough, increase at will (don't use '2048' as this would create needlessly thousands of TeX macros)
</code></pre></div>
<ul dir="auto">
<li>the second fix is to be found  in a <a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/scipy/scipy/commit/c881fdea7a7ef7c518b7898cfd53b2880af06778/hovercard" href="https://github.com/scipy/scipy/commit/c881fdea7a7ef7c518b7898cfd53b2880af06778">scipy/scipy@<tt>c881fde</tt></a> and I copy it here. It would thus be like this:</li>
</ul>
<div class="highlight highlight-text-tex-latex notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="% In the parameters section, place a newline after the Parameters
% header
\usepackage{expdlist}
\let\latexdescription=\description
\def\description{\latexdescription{}{} \breaklabel}

% Remove extra space
\usepackage{etoolbox}
\makeatletter
\patchcmd\@item{{\@breaklabel} }{{\@breaklabel}}{}{}
\makeatother

% Fix bug in expdlist's way of breaking the line after long item label
\makeatletter
\def\breaklabel{%
    \def\@breaklabel{%
        \leavevmode\par
        % now a hack because Sphinx inserts \leavevmode after term node
        \def\leavevmode{\def\leavevmode{\unhbox\voidb@x}}%
    }%
}
\makeatother"><pre class="notranslate"><span class="pl-c"><span class="pl-c">%</span> In the parameters section, place a newline after the Parameters</span>
<span class="pl-c"><span class="pl-c">%</span> header</span>
<span class="pl-k">\usepackage</span>{<span class="pl-c1">expdlist</span>}
<span class="pl-c1">\let\latexdescription</span>=<span class="pl-c1">\description</span>
<span class="pl-c1">\def\description</span>{<span class="pl-c1">\latexdescription</span>{}{} <span class="pl-c1">\breaklabel</span>}

<span class="pl-c"><span class="pl-c">%</span> Remove extra space</span>
<span class="pl-k">\usepackage</span>{<span class="pl-c1">etoolbox</span>}
<span class="pl-c1">\makeatletter</span>
<span class="pl-c1">\patchcmd\@item</span>{{<span class="pl-c1">\@breaklabel</span>} }{{<span class="pl-c1">\@breaklabel</span>}}{}{}
<span class="pl-c1">\makeatother</span>

<span class="pl-c"><span class="pl-c">%</span> Fix bug in expdlist's way of breaking the line after long item label</span>
<span class="pl-c1">\makeatletter</span>
<span class="pl-c1">\def\breaklabel</span>{<span class="pl-c"><span class="pl-c">%</span></span>
    <span class="pl-c1">\def\@breaklabel</span>{<span class="pl-c"><span class="pl-c">%</span></span>
        <span class="pl-c1">\leavevmode\par</span>
        <span class="pl-c"><span class="pl-c">%</span> now a hack because Sphinx inserts \leavevmode after term node</span>
        <span class="pl-c1">\def\leavevmode</span>{<span class="pl-c1">\def\leavevmode</span>{<span class="pl-c1">\unhbox\voidb@x</span>}}<span class="pl-c"><span class="pl-c">%</span></span>
    }<span class="pl-c"><span class="pl-c">%</span></span>
}
<span class="pl-c1">\makeatother</span></pre></div>
<p dir="auto">Side remark about<br>
</p><div class="Box Box--condensed my-2">
  <div class="Box-header f6">
    <p class="mb-0 text-bold">
      <a href="https://github.com/numpy/numpy/blob/b83f10ef7ee766bf30ccfa563b6cc8f7fd38a4c8/doc/source/conf.py#L182-L183">numpy/doc/source/conf.py</a>
    </p>
    <p class="mb-0 color-fg-muted">
        Lines 182 to 183
      in
      <a data-pjax="true" class="commit-tease-sha" href="/numpy/numpy/commit/b83f10ef7ee766bf30ccfa563b6cc8f7fd38a4c8">b83f10e</a>
    </p>
  </div>
  <div itemprop="text" class="Box-body p-0 blob-wrapper blob-wrapper-embedded data">
    <table class="highlight tab-size mb-0 js-file-line-container" data-tab-size="8" data-paste-markdown-skip="">

        <tbody><tr class="border-0">
          <td id="L182" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="182"></td>
          <td id="LC182" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-s">\usepackage{amsmath}</span> </td>
        </tr>

        <tr class="border-0">
          <td id="L183" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="183"></td>
          <td id="LC183" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-s">\DeclareUnicodeCharacter{00A0}{\nobreakspace}</span> </td>
        </tr>
    </tbody></table>
  </div>
</div>
<p></p>
<p dir="auto">It is unneeded because already done by Sphinx</p>
<p dir="auto">As for the remainder of the LaTeX code in this former <code class="notranslate">latex_preamble</code> it may need some check that it still gives a desired effect in the PDF.</p>
<p dir="auto">By the way I would recommend using <code class="notranslate">--halt-on-error</code> for <code class="notranslate">pdflatex</code> builds. In case of an error, the <code class="notranslate">make</code> exits with error code but the LaTeX console output is gigantic and the log file too and one loses time searching for first TeX error, whereas with <code class="notranslate">--halt-on-error</code> this is more efficient.</p>
<p dir="auto">The<br>
</p><div class="Box Box--condensed my-2">
  <div class="Box-header f6">
    <p class="mb-0 text-bold">
      <a href="https://github.com/numpy/numpy/blob/b83f10ef7ee766bf30ccfa563b6cc8f7fd38a4c8/doc/Makefile#L220">numpy/doc/Makefile</a>
    </p>
    <p class="mb-0 color-fg-muted">
         Line 220
      in
      <a data-pjax="true" class="commit-tease-sha" href="/numpy/numpy/commit/b83f10ef7ee766bf30ccfa563b6cc8f7fd38a4c8">b83f10e</a>
    </p>
  </div>
  <div itemprop="text" class="Box-body p-0 blob-wrapper blob-wrapper-embedded data">
    <table class="highlight tab-size mb-0 js-file-line-container" data-tab-size="8" data-paste-markdown-skip="">

        <tbody><tr class="border-0">
          <td id="L220" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="220"></td>
          <td id="LC220" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> perl -pi -e <span class="pl-s"><span class="pl-pds">'</span>s/\t(latex.*|pdflatex) (.*)/\t-$$1 -interaction batchmode $$2/<span class="pl-pds">'</span></span> build/latex/Makefile </td>
        </tr>
    </tbody></table>
  </div>
</div>
<br>
does nothing actually with current Sphinx produced Makefile in latex build directory. You can use the method from <a href="https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.latex.LaTeXBuilder" rel="nofollow">https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.latex.LaTeXBuilder</a><p></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="To pass options directly to the (pdf|xe|lua)latex binary, use variable LATEXOPTS, for example:

make latexpdf LATEXOPTS=&quot;--halt-on-error&quot;
"><pre class="notranslate"><code class="notranslate">To pass options directly to the (pdf|xe|lua)latex binary, use variable LATEXOPTS, for example:

make latexpdf LATEXOPTS="--halt-on-error"

</code></pre></div>
<p dir="auto">and</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Since 1.6, make latexpdf uses latexmk (not on Windows). This makes sure the needed number of runs is automatically executed to get the cross-references, bookmarks, indices, and tables of contents right.

One can pass to latexmk options via the LATEXMKOPTS Makefile variable. For example:

make latexpdf LATEXMKOPTS=&quot;-silent&quot;
"><pre class="notranslate"><code class="notranslate">Since 1.6, make latexpdf uses latexmk (not on Windows). This makes sure the needed number of runs is automatically executed to get the cross-references, bookmarks, indices, and tables of contents right.

One can pass to latexmk options via the LATEXMKOPTS Makefile variable. For example:

make latexpdf LATEXMKOPTS="-silent"

</code></pre></div>
<p dir="auto">(this is with standard Sphinx distributed Makefile, in your case replace <code class="notranslate">latexpdf</code> by  <code class="notranslate">-C build/latex all-pdf</code>) and the two can be combined but then (with patches of <code class="notranslate">expdlist</code> above commented out) the LaTeX error does not make it to the console output only to the end of the log file:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ make numpy-ref.pdf LATEXOPTS=&quot;--halt-on-error&quot; LATEXMKOPTS=&quot;--silent&quot;
latexmk -pdf -dvi- -ps- --silent 'numpy-ref.tex'
Latexmk: Run number 1 of rule 'makeindex numpy-ref.idx'
Latexmk: Run number 1 of rule 'pdflatex'
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (TeX Live 2019) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
Latexmk: Summary of warnings from last run of (pdf)latex:
  Latex found 4 multiply defined reference(s)
  Latex failed to resolve 388 citation(s)
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'numpy-ref.log' for details
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs, or warnings treated as errors.
make: *** [Makefile:33: numpy-ref.pdf] Error 12"><pre class="notranslate"><code class="notranslate">$ make numpy-ref.pdf LATEXOPTS="--halt-on-error" LATEXMKOPTS="--silent"
latexmk -pdf -dvi- -ps- --silent 'numpy-ref.tex'
Latexmk: Run number 1 of rule 'makeindex numpy-ref.idx'
Latexmk: Run number 1 of rule 'pdflatex'
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (TeX Live 2019) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
Latexmk: Summary of warnings from last run of (pdf)latex:
  Latex found 4 multiply defined reference(s)
  Latex failed to resolve 388 citation(s)
Collected error summary (may duplicate other messages):
  pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'numpy-ref.log' for details
Latexmk: Use the -f option to force complete processing,
 unless error was exceeding maximum runs, or warnings treated as errors.
make: *** [Makefile:33: numpy-ref.pdf] Error 12
</code></pre></div>
<p dir="auto">One has to go in <code class="notranslate">numpy-ref.log</code> find the LaTeX error causing abort:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="! Extra }, or forgotten \endgroup.
\@endpbox ...\@arstrutbox \color@endgroup \egroup 
                                                  \the \LT@p@ftn \global \LT...
l.131952 \item[{Parameters}]
                             \leavevmode 
Here is how much of TeX's memory you used:
 26928 strings out of 493377
 686470 string characters out of 6157389
 755139 words of memory out of 5000000
 26189 multiletter control sequences out of 15000+600000
 84239 words of font info for 133 fonts, out of 8000000 for 9000
 500 hyphenation exceptions out of 8191
 51i,26n,49p,1225b,732s stack positions out of 5000i,500n,10000p,200000b,80000s

!  ==&gt; Fatal error occurred, no output PDF file produced!"><pre class="notranslate"><code class="notranslate">! Extra }, or forgotten \endgroup.
\@endpbox ...\@arstrutbox \color@endgroup \egroup 
                                                  \the \LT@p@ftn \global \LT...
l.131952 \item[{Parameters}]
                             \leavevmode 
Here is how much of TeX's memory you used:
 26928 strings out of 493377
 686470 string characters out of 6157389
 755139 words of memory out of 5000000
 26189 multiletter control sequences out of 15000+600000
 84239 words of font info for 133 fonts, out of 8000000 for 9000
 500 hyphenation exceptions out of 8191
 51i,26n,49p,1225b,732s stack positions out of 5000i,500n,10000p,200000b,80000s

!  ==&gt; Fatal error occurred, no output PDF file produced!
</code></pre></div>
<p dir="auto">To recap, <code class="notranslate">numpy-ref.pdf</code> builds currently because <code class="notranslate">latex_preamble</code> from doc/source/conf.py is ignored! If this LaTeX configuration was not ignored, the build would fail due to usage of very old (1999) LaTeX package <code class="notranslate">expdlist</code> which causes usage of description lists in table cells break LaTeX (and there is one such usage subsection <code class="notranslate">distutils.misc</code> from section <code class="notranslate">Modules in numpy.distuils</code> part of <code class="notranslate">numpy-ref</code>). A fix is known from an April 2018 scipy commit <a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/scipy/scipy/commit/c881fde/hovercard" href="https://github.com/scipy/scipy/commit/c881fde">scipy/scipy@<tt>c881fde</tt></a>.</p>