<p dir="auto">xref <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="95119549" data-permission-text="Title is private" data-url="https://github.com/pandas-dev/pandas/issues/10577" data-hovercard-type="pull_request" data-hovercard-url="/pandas-dev/pandas/pull/10577/hovercard" href="https://github.com/pandas-dev/pandas/pull/10577">#10577</a> (has test for duplicates with empty data)</p>
<p dir="auto">I don't expect this is the correct behavior, although it's always possible I'm doing something wrong. Importing data using the <code class="notranslate">names</code> keyword will clobber the values of columns where the name is duplicated. For example:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from StringIO import StringIO
import pandas as pd

data = &quot;&quot;&quot;a,1
b,2
c,3&quot;&quot;&quot;
names = ['field', 'field']

print pd.read_csv(StringIO(data), names=names, mangle_dupe_cols=True)
print pd.read_csv(StringIO(data), names=names, mangle_dupe_cols=False)"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-v">StringIO</span> <span class="pl-k">import</span> <span class="pl-v">StringIO</span>
<span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>

<span class="pl-s1">data</span> <span class="pl-c1">=</span> <span class="pl-s">"""a,1</span>
<span class="pl-s">b,2</span>
<span class="pl-s">c,3"""</span>
<span class="pl-s1">names</span> <span class="pl-c1">=</span> [<span class="pl-s">'field'</span>, <span class="pl-s">'field'</span>]

<span class="pl-k">print</span> <span class="pl-s1">pd</span>.<span class="pl-en">read_csv</span>(<span class="pl-v">StringIO</span>(<span class="pl-s1">data</span>), <span class="pl-s1">names</span><span class="pl-c1">=</span><span class="pl-s1">names</span>, <span class="pl-s1">mangle_dupe_cols</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
<span class="pl-k">print</span> <span class="pl-s1">pd</span>.<span class="pl-en">read_csv</span>(<span class="pl-v">StringIO</span>(<span class="pl-s1">data</span>), <span class="pl-s1">names</span><span class="pl-c1">=</span><span class="pl-s1">names</span>, <span class="pl-s1">mangle_dupe_cols</span><span class="pl-c1">=</span><span class="pl-c1">False</span>)</pre></div>
<p dir="auto">returns</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="   field  field
0      1      1
1      2      2
2      3      3
   field  field
0      1      1
1      2      2
2      3      3"><pre class="notranslate"><code class="notranslate">   field  field
0      1      1
1      2      2
2      3      3
   field  field
0      1      1
1      2      2
2      3      3
</code></pre></div>
<p dir="auto">However, this produces the correct result:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="df = pd.read_csv(StringIO(data), header=None)
df.columns = names
print df"><pre class="notranslate"><span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-en">read_csv</span>(<span class="pl-v">StringIO</span>(<span class="pl-s1">data</span>), <span class="pl-s1">header</span><span class="pl-c1">=</span><span class="pl-c1">None</span>)
<span class="pl-s1">df</span>.<span class="pl-s1">columns</span> <span class="pl-c1">=</span> <span class="pl-s1">names</span>
<span class="pl-k">print</span> <span class="pl-s1">df</span></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="   field  field
0      a      1
1      b      2
2      c      3"><pre class="notranslate"><code class="notranslate">   field  field
0      a      1
1      b      2
2      c      3
</code></pre></div>
<p dir="auto">Interestingly, it works if the field names are in the header:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="data_with_header = &quot;field,field\n&quot; + data
print pd.read_csv(StringIO(data_with_header))"><pre class="notranslate"><span class="pl-s1">data_with_header</span> <span class="pl-c1">=</span> <span class="pl-s">"field,field<span class="pl-cce">\n</span>"</span> <span class="pl-c1">+</span> <span class="pl-s1">data</span>
<span class="pl-k">print</span> <span class="pl-s1">pd</span>.<span class="pl-en">read_csv</span>(<span class="pl-v">StringIO</span>(<span class="pl-s1">data_with_header</span>))</pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="  field  field.1
0     a        1
1     b        2
2     c        3"><pre class="notranslate"><code class="notranslate">  field  field.1
0     a        1
1     b        2
2     c        3
</code></pre></div>
<p dir="auto">Is this a bug or am I doing something wrong?</p>
<p dir="auto">I find R's <code class="notranslate">expand.grid()</code> function quite useful for quick creation of example datasets. For example:</p>
<div class="highlight highlight-source-r notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="expand.grid(height = seq(60, 70, 5), weight = seq(100, 180, 40), sex = c(&quot;Male&quot;,&quot;Female&quot;))"><pre class="notranslate">expand.grid(<span class="pl-v">height</span> <span class="pl-k">=</span> seq(<span class="pl-c1">60</span>, <span class="pl-c1">70</span>, <span class="pl-c1">5</span>), <span class="pl-v">weight</span> <span class="pl-k">=</span> seq(<span class="pl-c1">100</span>, <span class="pl-c1">180</span>, <span class="pl-c1">40</span>), <span class="pl-v">sex</span> <span class="pl-k">=</span> c(<span class="pl-s"><span class="pl-pds">"</span>Male<span class="pl-pds">"</span></span>,<span class="pl-s"><span class="pl-pds">"</span>Female<span class="pl-pds">"</span></span>))</pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="   height weight    sex
1      60    100   Male
2      65    100   Male
3      70    100   Male
4      60    140   Male
5      65    140   Male
6      70    140   Male
7      60    180   Male
8      65    180   Male
9      70    180   Male
10     60    100 Female
11     65    100 Female
12     70    100 Female
13     60    140 Female
14     65    140 Female
15     70    140 Female
16     60    180 Female
17     65    180 Female
18     70    180 Female"><pre class="notranslate"><code class="notranslate">   height weight    sex
1      60    100   Male
2      65    100   Male
3      70    100   Male
4      60    140   Male
5      65    140   Male
6      70    140   Male
7      60    180   Male
8      65    180   Male
9      70    180   Male
10     60    100 Female
11     65    100 Female
12     70    100 Female
13     60    140 Female
14     65    140 Female
15     70    140 Female
16     60    180 Female
17     65    180 Female
18     70    180 Female
</code></pre></div>
<p dir="auto">A simple implementation of this for <code class="notranslate">pandas</code> is easy to put together:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="def expand_grid(dct):
    rows = itertools.product(*dct.values())
    return pd.DataFrame.from_records(rows, columns=dct.keys())

df = expand_grid(
    {'height': range(60, 71, 5),
     'weight': range(100, 181, 40),
     'sex': ['Male', 'Female']}
)
print(df)"><pre class="notranslate"><span class="pl-k">def</span> <span class="pl-en">expand_grid</span>(<span class="pl-s1">dct</span>):
    <span class="pl-s1">rows</span> <span class="pl-c1">=</span> <span class="pl-s1">itertools</span>.<span class="pl-en">product</span>(<span class="pl-c1">*</span><span class="pl-s1">dct</span>.<span class="pl-en">values</span>())
    <span class="pl-k">return</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>.<span class="pl-en">from_records</span>(<span class="pl-s1">rows</span>, <span class="pl-s1">columns</span><span class="pl-c1">=</span><span class="pl-s1">dct</span>.<span class="pl-en">keys</span>())

<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-en">expand_grid</span>(
    {<span class="pl-s">'height'</span>: <span class="pl-en">range</span>(<span class="pl-c1">60</span>, <span class="pl-c1">71</span>, <span class="pl-c1">5</span>),
     <span class="pl-s">'weight'</span>: <span class="pl-en">range</span>(<span class="pl-c1">100</span>, <span class="pl-c1">181</span>, <span class="pl-c1">40</span>),
     <span class="pl-s">'sex'</span>: [<span class="pl-s">'Male'</span>, <span class="pl-s">'Female'</span>]}
)
<span class="pl-en">print</span>(<span class="pl-s1">df</span>)</pre></div>
<p dir="auto">Do people think this would be a useful addition?</p>
<p dir="auto">If so, what kind of features should it have beyond the basics? A <code class="notranslate">dtypes</code> argument, specifying which column should be the index, etc.?</p>
<p dir="auto">I'm also not sure if <code class="notranslate">expand_grid</code> is the most intuitive name, but given that it's duplicating<br>
R functionality, maybe it's best just to leave it as is.</p>