<p dir="auto"><em>Original ticket <a href="http://projects.scipy.org/numpy/ticket/872" rel="nofollow">http://projects.scipy.org/numpy/ticket/872</a> on 2008-08-04 by trac user novak, assigned to unknown.</em></p>
<p dir="auto">I believe that setmember1d contains unnecessary code.  Also, the docstring of setmember1d asserts that the arguments must not contain duplicated values, but I believe that this is incorrect.  The function works as expected even with duplication.  Below is my post to the numpy-discussion mailing list to this effect.</p>
<hr>
<p dir="auto">I have two arrays of integers, and would like to know <em>where</em> they<br>
have elements in common, not just <em>which</em> elements are in common.<br>
This is because the entries in the integer array are aligned with<br>
other arrays.  This seems very close to what member1d advertises as<br>
its function.  However, member1d says that it expects arrays with only<br>
unique elements.</p>
<p dir="auto">First of all, my desired operation is well-posed:  I'd like f(ar1,<br>
ar2) to return something in the shape of ar1 with True if the value at<br>
that position appears anywhere in ar2 (regardless of duplication) and<br>
False otherwise.</p>
<p dir="auto">So I looked at the code and have two questions:</p>
<ol dir="auto">
<li>What is this code trying to achieve?<br>
aux = perm[ii+1]<br>
perm[ii+1] = perm[ii]<br>
perm[ii] = aux</li>
</ol>
<p dir="auto">Here perm is the stable argsort of the two concatenated arguments:<br>
perm = concatenate((ar1, ar2)).argsort(kind='mergesort').<br>
arr is the array of combined inputs in sorted order:<br>
arr = concatenate((ar1, ar2))[perm]<br>
and ii is a list of indices into arr where the value of arr is equal<br>
to the next value in the array (arr[ii] == arr[ii+1]) <em>and</em> arr[ii]<br>
came from the <em>second</em> input (ar2).</p>
<p dir="auto">Now, this last bit (looking for elements of arr that are equal and<br>
both came from the second array) is clearly trying to deal with<br>
duplication, which is why I'm interested...</p>
<p dir="auto">So, the code snippet is trying to swap perm[ii+1] with perm[ii], but I<br>
don't see why.  Furthermore, there are funny results if a value is<br>
duplicated three times, not just twice -- perm is no longer a<br>
permutation vector.  Eg, member1d([1], [2,2,2]) results perm=[0,1,2,3]<br>
and ii=[1,2] before the above snippet, and the above snippet makes<br>
perm into [0,2,3,2]</p>
<p dir="auto">I've commented those three lines, and I've never seen any changes to<br>
the output of member1d.  The new value of perm is used to compute the<br>
expression: perm.argsort(kind='mergesort')[:len( ar1 )], but the<br>
changes to that expression as a result of the above three lines are<br>
always at the high end of the array, which is sliced off by the last<br>
[:len(ar1)].</p>
<p dir="auto">Finally, my second question is:<br>
2) Does anyone have a test case where member1d fails as a result of<br>
duplicates in the input?  So far I haven't found any, with the above<br>
lines commented or not.</p>
<p dir="auto">Upon reflection and review of the changelog, another theory occurs to<br>
me: member1d did not originally use a stable sort.  What I've written<br>
above for interpretation of the value ii (indicates duplication within<br>
ar2) is true for a stable sort, but for an unstable sort the same<br>
condition has the interpretation that ii holds the values where the<br>
sorting algorithm swapped the order of equal values unstably.  Then<br>
the code snippet in question 1) looks like an attempt to swap those<br>
values in the permutation array to make the sort stable again.  The<br>
attempt would fail if there was duplication in either array.</p>
<p dir="auto">So, I would propose deleting those three lines (since they seem to be<br>
a non-functional relic) and declaring in the docstring that member1d<br>
doesn't require unique elements.</p>
<p dir="auto">Also, if this is correct, then the function simplifies considerably<br>
since several values don't need to be computed anymore:</p>
<p dir="auto">def setmember1d( ar1, ar2 ):<br>
ar = nm.concatenate( (ar1, ar2 ) )<br>
perm = ar.argsort(kind='mergesort')<br>
aux = ar[perm]<br>
flag = nm.concatenate( (aux[1:] == aux[:-1], [False] ) )<br>
indx = perm.argsort(kind='mergesort')[:len( ar1 )]<br>
return flag[indx]</p>
<p dir="auto">np version 1.9.2 (but bug in previous versions as well)<br>
python version 3.4.3 (but bug with previous versions as well)</p>
<p dir="auto">The following demonstrates the problem:<br>
On 64Bit architecture:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; import numpy as np
&gt;&gt;&gt; np.intp
&lt;class 'numpy.int64'&gt;
&gt;&gt;&gt; np.intp==np.int64
True"><pre class="notranslate"><span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">np</span>.<span class="pl-s1">intp</span>
<span class="pl-c1">&lt;</span><span class="pl-k">class</span> <span class="pl-s">'numpy.int64'</span><span class="pl-c1">&gt;</span>
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">np</span>.<span class="pl-s1">intp</span><span class="pl-c1">==</span><span class="pl-s1">np</span>.<span class="pl-s1">int64</span>
<span class="pl-c1">True</span></pre></div>
<p dir="auto">On 32Bit architecture:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="&gt;&gt;&gt; import numpy as np
&gt;&gt;&gt; np.intp
&lt;class 'numpy.int32'&gt;
&gt;&gt;&gt; np.intp==np.int32
False"><pre class="notranslate"><span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">np</span>.<span class="pl-s1">intp</span>
<span class="pl-c1">&lt;</span><span class="pl-k">class</span> <span class="pl-s">'numpy.int32'</span><span class="pl-c1">&gt;</span>
<span class="pl-c1">&gt;&gt;</span><span class="pl-c1">&gt;</span> <span class="pl-s1">np</span>.<span class="pl-s1">intp</span><span class="pl-c1">==</span><span class="pl-s1">np</span>.<span class="pl-s1">int32</span>
<span class="pl-c1">False</span></pre></div>