<p dir="auto">This issue is an exact analog of <a href="https://github.com/numpy/numpy/issues/3733" data-hovercard-type="issue" data-hovercard-url="/numpy/numpy/issues/3733/hovercard">numpy 3733</a>, because the code between these this function and <code class="notranslate">np.percentile</code> is mostly duplicated.</p>
<p dir="auto"><code class="notranslate">scoreatpercentile</code> claims to function with the <code class="notranslate">per</code> argument (the percentiles to calculate) being either a scalar or an array_like. However, the initial lines of code do:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="if per == 0:
    return a.min(axis=axis, out=out)
elif per == 100:
    return a.max(axis=axis, out=out)"><pre class="notranslate"><code class="notranslate">if per == 0:
    return a.min(axis=axis, out=out)
elif per == 100:
    return a.max(axis=axis, out=out)
</code></pre></div>
<p dir="auto">which of course breaks with a "truth value of array undefined" exception if <code class="notranslate">per</code> is in fact an ndarray. It doesn't break if <code class="notranslate">per</code> is a list, since <code class="notranslate">per</code> isn't coerced to array-ness, but that's just two bugs canceling.</p>
<p dir="auto">From an initial reading, I think the implementations of these two functions could be merged. <code class="notranslate">np.concatenate</code> could be given an <code class="notranslate">interpolate_method</code> argument like <code class="notranslate">scoreatpercentile</code>, and then <code class="notranslate">scipy.stats.scoreatpercentile</code> could just become an alias for the Numpy function.</p>
<p dir="auto">The discussion started in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="75641955" data-permission-text="Title is private" data-url="https://github.com/scipy/scipy/issues/4864" data-hovercard-type="issue" data-hovercard-url="/scipy/scipy/issues/4864/hovercard" href="https://github.com/scipy/scipy/issues/4864">#4864</a> . I propose that the usage of <code class="notranslate">polish</code> in the <code class="notranslate">_differentialevolution.py</code> must be deprecated. The polish functionality simply repeats the functionality of other <code class="notranslate">scipy.optimize</code> module. The library functions must be simple, universal and not specialised and overfeatured. The polish can be applied to the result externally by a user using <code class="notranslate">optimize.minimize</code> with the same cost, but <code class="notranslate">_differentialevolution.py</code> will be more universal, will have one less dependency and cleaner code.</p>
<p dir="auto">Additional reasoning:<br>
The function is called <code class="notranslate">differential_evolution</code> and not a <code class="notranslate">differential_evolution_with_polishing</code>. I would not expect from this function to do something else from DE. OK, if there is statistics that this polishing really goes in majority of real world cases, then let leave it. But, at least the default behavior should be changed to <code class="notranslate">polish = False</code>. (Are there some benchmarks about comparing what is more efficient -- letting a couple of additional iterations of DE, or applying polishing? What about the type of the testfunction?)<br>
Additionally, this polisher, I hypothesize, works OK only with differentiable functions. But there are a lot of real engineering problems (my among them), where objective functions are not only nonlinear, but also are discontinuous and non-differentiable and somewhat noizy, and where the global minimum can be just near the discontinuity. I still prefer modularity, and I belive, that all library functions should be modular and perform good just one task.</p>
<p dir="auto">So, this additional feature could be removed with mentioning in documentation, or in the examples, that <code class="notranslate">L-BFGS-B</code> polishing is usually best suited and gives good results for differentiable smooth objective functions. Or, at least, the default behaviour should be changed to <code class="notranslate">polish = False</code>.</p>