Question: I am trying to remove duplicates values of two cols in a df - I don't get an error, but the code runs but does not remove the dups -

[![enter image description here](https://i.stack.imgur.com/7qSQ7.png)](https://i.stack.imgur.com/7qSQ7.png)
first I run this below function to remove all possible duplicate combinations like below - 

```
def remove_dup_combos(df):
u = df.filter(like='id').values
m = pd.DataFrame(np.sort(u, axis=1)).duplicated()
df = df[~m]

return df

```

This removes all duplicate combos 

[![enter image description here](https://i.stack.imgur.com/Hsq4J.png)](https://i.stack.imgur.com/Hsq4J.png)

I now try to remove all dups in both id cols with 

```
def remove_dups(df):
df = df.drop_duplicates(['id1', 'id2'],keep='first')

return df

```

but this does not give me expetcted output and the output is same as second output above

thanks !

Answer: 
It seems that you want duplicates to be determined on the basis of the *unordered* id, which Pandas doesn't have a sense of. For example, these two entries

```
df = pd.DataFrame([['a', 'b'], ['b', 'a']], columns=['id1', 'id2'])

  id1 id2
0   a   b
1   b   a

```

would considered different because they don't match on `id1` or `id2`. If you want to check whether the unordered set of both `id`s is a duplicate, you can compare whether their minimum and their maximum is a duplicate. For example, adding some temporary columns,

```
df.assign(min_id = lambda x : x[['id1', 'id2']].min(axis=1),
          max_id = lambda x : x[['id1', 'id2']].max(axis=1)) \
    .drop_duplicates(subset=['min_id', 'max_id']) \
    .drop(['min_id', 'max_id'], axis=1)

```

which is pretty much what the NumPy solution in the question is doing.
If need remove same values per columns:

```
df = df[df['id1'] != df['id2']]

```