<p dir="auto">There is an asymmetry in Pandas wrt. indexes vs. columns that is not present in the relational model. For example, for the purposes of merging one has to decide whether the column on which to merge is an index or not, despite the fact that indexing is incidental complexity for optimization. This leads to many of us having duplicate indexes and columns with the same name, which can cause errors and wastes memory for large data frames.</p>
<p dir="auto">A possible fix would be to address the column lookup in all the current operations that require columns or indexes so that on NameError for columns the named indexes would be fetched instead. Or one may as well go all the way relational and make <em>any</em> <code class="notranslate">df[my_col_or_named_index]</code> lookup fetch a named index if the column name is not found.</p>
<p dir="auto">The latter behaviour is being implemented in the eval/query functionality by <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jreback/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jreback">@jreback</a> and <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/cpcloud/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/cpcloud">@cpcloud</a> in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="16492323" data-permission-text="Title is private" data-url="https://github.com/pandas-dev/pandas/issues/4164" data-hovercard-type="pull_request" data-hovercard-url="/pandas-dev/pandas/pull/4164/hovercard" href="https://github.com/pandas-dev/pandas/pull/4164">#4164</a>, so it would also be 'least surprise' to use this policy across the board.</p>
<h4 dir="auto">Code Sample, a copy-pastable example if possible</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import pandas
df = pandas.DataFrame(columns=['A', 'B'])
print(df.columns)
df = df.filter(items=['A', 'A', 'A'])
print(df.columns)"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">pandas</span>
<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pandas</span>.<span class="pl-v">DataFrame</span>(<span class="pl-s1">columns</span><span class="pl-c1">=</span>[<span class="pl-s">'A'</span>, <span class="pl-s">'B'</span>])
<span class="pl-en">print</span>(<span class="pl-s1">df</span>.<span class="pl-s1">columns</span>)
<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">df</span>.<span class="pl-en">filter</span>(<span class="pl-s1">items</span><span class="pl-c1">=</span>[<span class="pl-s">'A'</span>, <span class="pl-s">'A'</span>, <span class="pl-s">'A'</span>])
<span class="pl-en">print</span>(<span class="pl-s1">df</span>.<span class="pl-s1">columns</span>)</pre></div>
<h4 dir="auto">Problem description</h4>
<p dir="auto">Filtering a dataframe with a list which contains multiple times a column that is already inside the original dataframe results in a new dataframe, where the mentioned column is present multiple times. This problem might be fixed easily by converting the input list into a set.</p>
<h4 dir="auto">Expected Output</h4>
<p dir="auto">A filtered dataframe which have only one column: 'A'.</p>
<h4 dir="auto">Output of <code class="notranslate">pd.show_versions()</code></h4>
<details>
<h2 dir="auto">[paste the output of <code class="notranslate">pd.show_versions()</code> here below this line]<br>
INSTALLED VERSIONS</h2>
<p dir="auto">commit: None<br>
python: 3.6.3.final.0<br>
python-bits: 64<br>
OS: Windows<br>
OS-release: 10<br>
machine: AMD64<br>
processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel<br>
byteorder: little<br>
LC_ALL: None<br>
LANG: None<br>
LOCALE: None.None</p>
<p dir="auto">pandas: 0.22.0<br>
pytest: None<br>
pip: 9.0.1<br>
setuptools: 28.8.0<br>
Cython: None<br>
numpy: 1.14.3<br>
scipy: None<br>
pyarrow: None<br>
xarray: None<br>
IPython: 6.2.1<br>
sphinx: None<br>
patsy: None<br>
dateutil: 2.7.3<br>
pytz: 2018.4<br>
blosc: None<br>
bottleneck: None<br>
tables: None<br>
numexpr: None<br>
feather: None<br>
matplotlib: None<br>
openpyxl: None<br>
xlrd: None<br>
xlwt: None<br>
xlsxwriter: None<br>
lxml: None<br>
bs4: None<br>
html5lib: 1.0b10<br>
sqlalchemy: 1.2.3<br>
pymysql: None<br>
psycopg2: None<br>
jinja2: 2.9.6<br>
s3fs: None<br>
fastparquet: None<br>
pandas_gbq: None<br>
pandas_datareader: None</p>
</details>