<p dir="auto">related <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="17220085" data-permission-text="Title is private" data-url="https://github.com/pandas-dev/pandas/issues/4363" data-hovercard-type="issue" data-hovercard-url="/pandas-dev/pandas/issues/4363/hovercard" href="https://github.com/pandas-dev/pandas/issues/4363">#4363</a><br>
closes <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="13257105" data-permission-text="Title is private" data-url="https://github.com/pandas-dev/pandas/issues/3374" data-hovercard-type="issue" data-hovercard-url="/pandas-dev/pandas/issues/3374/hovercard" href="https://github.com/pandas-dev/pandas/issues/3374">#3374</a></p>
<p dir="auto">Here is a minimal example:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [21]: df = pd.DataFrame({'A': range(5), 'B': rand(5)})

In [22]: df
Out[22]: 
   A         B
0  0  0.402616
1  1  0.880696
2  2  0.184491
3  3  0.832732
4  4  0.393917

[5 rows x 2 columns]

In [23]: df.to_csv('test.csv', sep=' ')

In [24]: pd.read_csv('test.csv', sep=' ', dtype={'A': np.float64}).dtypes
Out[24]: 
Unnamed: 0      int64
A             float64
B             float64
dtype: object"><pre lang="ipython" class="notranslate"><code class="notranslate">In [21]: df = pd.DataFrame({'A': range(5), 'B': rand(5)})

In [22]: df
Out[22]: 
   A         B
0  0  0.402616
1  1  0.880696
2  2  0.184491
3  3  0.832732
4  4  0.393917

[5 rows x 2 columns]

In [23]: df.to_csv('test.csv', sep=' ')

In [24]: pd.read_csv('test.csv', sep=' ', dtype={'A': np.float64}).dtypes
Out[24]: 
Unnamed: 0      int64
A             float64
B             float64
dtype: object
</code></pre></div>
<p dir="auto">Here the dtype argument behaves as expected, and column A has type float. However with sep='\s' the dtype argument appears to be ignored:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [25]: pd.read_csv('test.csv', sep='\s', dtype={'A': np.float64}).dtypes
Out[25]: 
A      int64
B    float64
dtype: object"><pre lang="ipython" class="notranslate"><code class="notranslate">In [25]: pd.read_csv('test.csv', sep='\s', dtype={'A': np.float64}).dtypes
Out[25]: 
A      int64
B    float64
dtype: object
</code></pre></div>
<h2 dir="auto">Version information</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [27]: show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Darwin
OS-release: 10.8.0
machine: i386
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.13.1-413-ga71ede3
Cython: 0.20.1
numpy: 1.8.0
scipy: 0.13.3
statsmodels: 0.5.0
IPython: 2.0.0-dev
sphinx: 1.2.1
patsy: 0.2.1
scikits.timeseries: None
dateutil: 1.5
pytz: 2013b
bottleneck: None
tables: 3.1.0
numexpr: 2.3.1
matplotlib: 1.3.1
openpyxl: 1.8.2
xlrd: 0.9.2
xlwt: 0.7.5
xlsxwriter: 0.5.2
lxml: 3.3.1
bs4: 4.3.1
html5lib: None
bq: None
apiclient: None
rpy2: None
sqlalchemy: 0.9.2
pymysql: None
psycopg2: None"><pre lang="ipython" class="notranslate"><code class="notranslate">In [27]: show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Darwin
OS-release: 10.8.0
machine: i386
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.13.1-413-ga71ede3
Cython: 0.20.1
numpy: 1.8.0
scipy: 0.13.3
statsmodels: 0.5.0
IPython: 2.0.0-dev
sphinx: 1.2.1
patsy: 0.2.1
scikits.timeseries: None
dateutil: 1.5
pytz: 2013b
bottleneck: None
tables: 3.1.0
numexpr: 2.3.1
matplotlib: 1.3.1
openpyxl: 1.8.2
xlrd: 0.9.2
xlwt: 0.7.5
xlsxwriter: 0.5.2
lxml: 3.3.1
bs4: 4.3.1
html5lib: None
bq: None
apiclient: None
rpy2: None
sqlalchemy: 0.9.2
pymysql: None
psycopg2: None
</code></pre></div>
<p dir="auto">While trying to get handle on duplicated records I stumble upon <a href="http://stackoverflow.com/questions/26244309/how-to-analyze-all-duplicate-entries-in-this-pandas-dataframe" rel="nofollow">this</a> which lead to conclusion that <code class="notranslate">.duplicated(take_last=True)</code> seem to be taking the first of duplicate rows and <code class="notranslate">.duplicate(take_last=False)</code> takes the last rows.<br>
Here is an illustration:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import pandas as pd
data = { 'key1':[1,2,3,1,2,3,2,2,2],
      'key2':[2,2,1,2,2,2,2,2,2],
      'dup':['d1_1','d2_1', 'n_d','d1_2','d2_2', 'n_d','d2_3','d2_4','d2_5']}
df = pd.DataFrame(data,columns=['key1','key2','dup'])
print df
   key1  key2  dup
0     1     2  d1_1
1     2     2  d2_1
2     3     1   n_d
3     1     2  d1_2
4     2     2  d2_2
5     3     2   n_d
6     2     2  d2_3
7     2     2  d2_4
8     2     2  d2_5"><pre class="notranslate"><code class="notranslate">import pandas as pd
data = { 'key1':[1,2,3,1,2,3,2,2,2],
      'key2':[2,2,1,2,2,2,2,2,2],
      'dup':['d1_1','d2_1', 'n_d','d1_2','d2_2', 'n_d','d2_3','d2_4','d2_5']}
df = pd.DataFrame(data,columns=['key1','key2','dup'])
print df
   key1  key2  dup
0     1     2  d1_1
1     2     2  d2_1
2     3     1   n_d
3     1     2  d1_2
4     2     2  d2_2
5     3     2   n_d
6     2     2  d2_3
7     2     2  d2_4
8     2     2  d2_5
</code></pre></div>
<p dir="auto">Now with <code class="notranslate">take_last=False</code> it would be fair to expect dn_1s to be in output, but this is not the case:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="c1 = df.duplicated(['key1', 'key2'], take_last=False)
df[c1]
   key1  key2   dup
3     1     2  d1_2
4     2     2  d2_2
6     2     2  d2_3
7     2     2  d2_4
8     2     2  d2_5"><pre class="notranslate"><code class="notranslate">c1 = df.duplicated(['key1', 'key2'], take_last=False)
df[c1]
   key1  key2   dup
3     1     2  d1_2
4     2     2  d2_2
6     2     2  d2_3
7     2     2  d2_4
8     2     2  d2_5
</code></pre></div>
<p dir="auto">And <code class="notranslate">take_last=True</code> outputs the first rows:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="c2 = df.duplicated(['key1', 'key2'], take_last=True)
df[c2]
   key1  key2   dup
0     1     2  d1_1
1     2     2  d2_1
4     2     2  d2_2
6     2     2  d2_3
7     2     2  d2_4"><pre class="notranslate"><code class="notranslate">c2 = df.duplicated(['key1', 'key2'], take_last=True)
df[c2]
   key1  key2   dup
0     1     2  d1_1
1     2     2  d2_1
4     2     2  d2_2
6     2     2  d2_3
7     2     2  d2_4
</code></pre></div>
<p dir="auto">Unless I am misunderstanding the doc:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="take_last : boolean, default False
    Take the last observed row in a row. Defaults to the first row"><pre class="notranslate"><code class="notranslate">take_last : boolean, default False
    Take the last observed row in a row. Defaults to the first row
</code></pre></div>
<p dir="auto">it does feel that <code class="notranslate">.duplicated()</code> could be improved by fixing this behavior.<br>
And additionally it would have one more parameter:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="take_all : boolean, default False
    Take all observed rows. Overrides take_last"><pre class="notranslate"><code class="notranslate">take_all : boolean, default False
    Take all observed rows. Overrides take_last
</code></pre></div>
<p dir="auto">or alternatively a keyword parameter:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="take : 'last', 'first', 'all', default 'last'
    Sets which observed duplicated rows to take. Default: take last observed rows."><pre class="notranslate"><code class="notranslate">take : 'last', 'first', 'all', default 'last'
    Sets which observed duplicated rows to take. Default: take last observed rows.
</code></pre></div>
<p dir="auto">Right now trying to get all observed duplicates requires applying two above conditions <code class="notranslate">c1 | c2</code>.<br>
This was done with pandas 0.14.1.<br>
Thank you.</p>