<p dir="auto">due to the use of apply_over_axis masked median operations gets incredibly slow when computing them along a small dimension of a large array:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [3]: d = np.random.uniform(size=(200, 200, 50))
In [4]: %timeit np.median(d,axis=2)
10 loops, best of 3: 40.1 ms per loop
In [5]: %timeit np.nanmedian(d,axis=2)
1 loops, best of 3: 1.32 s per loop
In [6]: %timeit np.ma.median(d,axis=2)
1 loops, best of 3: 4.89 s per loop"><pre class="notranslate"><code class="notranslate">In [3]: d = np.random.uniform(size=(200, 200, 50))
In [4]: %timeit np.median(d,axis=2)
10 loops, best of 3: 40.1 ms per loop
In [5]: %timeit np.nanmedian(d,axis=2)
1 loops, best of 3: 1.32 s per loop
In [6]: %timeit np.ma.median(d,axis=2)
1 loops, best of 3: 4.89 s per loop
</code></pre></div>
<p dir="auto">a way out is using bottlenecks nanmedian which is on par with our non-masked performance (and better for the slow axis) but I think we should think of ways to improve this in numpy.</p>
<p dir="auto">I can think of following improvements for median:</p>
<ul dir="auto">
<li>implement apply_over_axis in C, should be relatively easy but will only give us ~50% boost at best. it should still be worthwhile as there are likely more users of that function out there.</li>
<li>extend partition to broadcast the index, then one could compute the right index from the mask and do everything in one partition call. Probably worthwhile but it will make the partition interface more complicated and would probably imply rewritting our pretty ugly sorting code.</li>
<li>probably the simplest would be add a check on the size of the axis and use sort instead of partition as currently done in ma.median but then instead of selecting via apply_along_axis, one selects the elements to average (or not) via fancy indexing (always select the middle two and if the non-masked count is odd duplicate one of the elements so mean() does nothing)</li>
<li>add a MaskedPartition C-Api function</li>
<li>see if we can integrate bottleneck directly somehow</li>
</ul>
<p dir="auto">any more options I'm might be overlooking?</p>
<p dir="auto">We have two pages labeled "diversity statement":</p>
<ul dir="auto">
<li>
<p dir="auto">The new one: <a href="https://numpy.org/diversity_sep2020/" rel="nofollow">NumPy diversity and inclusion statement</a></p>
</li>
<li>
<p dir="auto">The one in the Code of Conduct: <a href="https://numpy.org/devdocs/dev/conduct/code_of_conduct.html#diversity-statement" rel="nofollow">Diversity Statement</a></p>
</li>
</ul>
<p dir="auto">Neither mentions the other.</p>
<ul dir="auto">
<li>Should the new statement have at least a link to the CoC diversity statement?</li>
<li>Should the new statement say "September 2020" in the title to indicate that it's issued at a particular point in time in response to events (unlike the CoC),  and will be updated?</li>
</ul>
<blockquote>
<h1 dir="auto">NumPy diversity and inclusion statement: September 2020</h1>
</blockquote>