<p dir="auto">Several puzzling behaviors appear below wherein calling <code class="notranslate">Series.iloc[int]</code> for a Series with a MultiIndex gives confusing results.  Namely, I see unexpected indices mutated.  Unexpected indices read and then added to others.  As well as more than one index being mutated.  Contrast all of the behaviors below with indexing a vanilla numpy array where none of the mystery occurs.  Full recreates below along with why I find them surprising.  I've seen similar behavior back in the Pandas 0.19.x series.</p>
<h4 dir="auto">Code Sample 1</h4>
<p dir="auto">The length-2 case with duplicated keys at the first level.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [1]: import pandas as pd

In [2]: z = pd.DataFrame()
   ...: z['i'] = [1, 1]
   ...: z['j'] = [22, 33]
   ...: z['k'] = [555, 666]
   ...: z.set_index(['i', 'j'], inplace=True)
   ...: v = z.k
   ...: 

In [3]: v
Out[3]: 
i  j 
1  22    555
   33    666
Name: k, dtype: int64

In [4]: v.values
Out[4]: array([555, 666])

In [5]: v.iloc[0] += 200; v
Out[5]: 
i  j 
1  22    755
   33    666
Name: k, dtype: int64

In [6]: v.iloc[-1] += 300; v
Out[6]: 
i  j 
1  22    755
   33    966
Name: k, dtype: int64

In [7]: v.iloc[1] += 100; v
Out[7]: 
i  j 
1  22    1066
   33    1066
Name: k, dtype: int64"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">1</span>]: <span class="pl-s1">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>

<span class="pl-v">In</span> [<span class="pl-c1">2</span>]: <span class="pl-s1">z</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>()
   ...: <span class="pl-s1">z</span>[<span class="pl-s">'i'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">1</span>, <span class="pl-c1">1</span>]
   ...: <span class="pl-s1">z</span>[<span class="pl-s">'j'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">22</span>, <span class="pl-c1">33</span>]
   ...: <span class="pl-s1">z</span>[<span class="pl-s">'k'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">555</span>, <span class="pl-c1">666</span>]
   ...: <span class="pl-s1">z</span>.<span class="pl-en">set_index</span>([<span class="pl-s">'i'</span>, <span class="pl-s">'j'</span>], <span class="pl-s1">inplace</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
   ...: <span class="pl-s1">v</span> <span class="pl-c1">=</span> <span class="pl-s1">z</span>.<span class="pl-s1">k</span>
   ...: 

<span class="pl-v">In</span> [<span class="pl-c1">3</span>]: <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">3</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span> 
<span class="pl-c1">1</span>  <span class="pl-c1">22</span>    <span class="pl-c1">555</span>
   <span class="pl-c1">33</span>    <span class="pl-c1">666</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">4</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">values</span>
<span class="pl-v">Out</span>[<span class="pl-c1">4</span>]: <span class="pl-en">array</span>([<span class="pl-c1">555</span>, <span class="pl-c1">666</span>])

<span class="pl-v">In</span> [<span class="pl-c1">5</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">0</span>] <span class="pl-c1">+=</span> <span class="pl-c1">200</span>; <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">5</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span> 
<span class="pl-c1">1</span>  <span class="pl-c1">22</span>    <span class="pl-c1">755</span>
   <span class="pl-c1">33</span>    <span class="pl-c1">666</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">6</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">-</span><span class="pl-c1">1</span>] <span class="pl-c1">+=</span> <span class="pl-c1">300</span>; <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">6</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span> 
<span class="pl-c1">1</span>  <span class="pl-c1">22</span>    <span class="pl-c1">755</span>
   <span class="pl-c1">33</span>    <span class="pl-c1">966</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">7</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">1</span>] <span class="pl-c1">+=</span> <span class="pl-c1">100</span>; <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">7</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span> 
<span class="pl-c1">1</span>  <span class="pl-c1">22</span>    <span class="pl-c1">1066</span>
   <span class="pl-c1">33</span>    <span class="pl-c1">1066</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span></pre></div>
<h4 dir="auto">Code Sample 1 Problem / Expected Result</h4>
<p dir="auto">Final statement incremented key (1, 33) then wrote the result to be (1, 22) and (1, 33)?<br>
Expected (1, 22) to remain 755 and (1, 33) to become 1066.</p>
<h4 dir="auto">Code Sample 2</h4>
<p dir="auto">The length-2 case with unique keys at each level.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [1]: import pandas as pd

In [2]: z = pd.DataFrame()
   ...: z['i'] = [1, 2]
   ...: z['j'] = [3, 4]
   ...: z['k'] = [7, 8]
   ...: z.set_index(['i', 'j'], inplace=True)
   ...: v = z.k
   ...: 

In [3]: v
Out[3]: 
i  j
1  3    7
2  4    8
Name: k, dtype: int64

In [4]: v.values
Out[4]: array([7, 8])

In [5]: v.iloc[0] += 10; v
Out[5]: 
i  j
1  3    17
2  4     8
Name: k, dtype: int64

In [6]: v.iloc[-1] += 10; v
Out[6]: 
i  j
1  3    17
2  4    18
Name: k, dtype: int64

In [7]: v.iloc[1] += 1000; v
Out[7]: 
i  j
1  3    1018
2  4      18
Name: k, dtype: int64"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">1</span>]: <span class="pl-s1">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>

<span class="pl-v">In</span> [<span class="pl-c1">2</span>]: <span class="pl-s1">z</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>()
   ...: <span class="pl-s1">z</span>[<span class="pl-s">'i'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">1</span>, <span class="pl-c1">2</span>]
   ...: <span class="pl-s1">z</span>[<span class="pl-s">'j'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">3</span>, <span class="pl-c1">4</span>]
   ...: <span class="pl-s1">z</span>[<span class="pl-s">'k'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">7</span>, <span class="pl-c1">8</span>]
   ...: <span class="pl-s1">z</span>.<span class="pl-en">set_index</span>([<span class="pl-s">'i'</span>, <span class="pl-s">'j'</span>], <span class="pl-s1">inplace</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
   ...: <span class="pl-s1">v</span> <span class="pl-c1">=</span> <span class="pl-s1">z</span>.<span class="pl-s1">k</span>
   ...: 

<span class="pl-v">In</span> [<span class="pl-c1">3</span>]: <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">3</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span>
<span class="pl-c1">1</span>  <span class="pl-c1">3</span>    <span class="pl-c1">7</span>
<span class="pl-c1">2</span>  <span class="pl-c1">4</span>    <span class="pl-c1">8</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">4</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">values</span>
<span class="pl-v">Out</span>[<span class="pl-c1">4</span>]: <span class="pl-en">array</span>([<span class="pl-c1">7</span>, <span class="pl-c1">8</span>])

<span class="pl-v">In</span> [<span class="pl-c1">5</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">0</span>] <span class="pl-c1">+=</span> <span class="pl-c1">10</span>; <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">5</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span>
<span class="pl-c1">1</span>  <span class="pl-c1">3</span>    <span class="pl-c1">17</span>
<span class="pl-c1">2</span>  <span class="pl-c1">4</span>     <span class="pl-c1">8</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">6</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">-</span><span class="pl-c1">1</span>] <span class="pl-c1">+=</span> <span class="pl-c1">10</span>; <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">6</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span>
<span class="pl-c1">1</span>  <span class="pl-c1">3</span>    <span class="pl-c1">17</span>
<span class="pl-c1">2</span>  <span class="pl-c1">4</span>    <span class="pl-c1">18</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">7</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">1</span>] <span class="pl-c1">+=</span> <span class="pl-c1">1000</span>; <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">7</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span>
<span class="pl-c1">1</span>  <span class="pl-c1">3</span>    <span class="pl-c1">1018</span>
<span class="pl-c1">2</span>  <span class="pl-c1">4</span>      <span class="pl-c1">18</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span></pre></div>
<h4 dir="auto">Code Sample 2 Problem / Expected Result</h4>
<p dir="auto">Above iloc[0] incremented key (1, 3) but then iloc[1] does?<br>
Do not expect 2 different iloc indices differing by 1 to point to the same location in a length 2 array.</p>
<h4 dir="auto">Code Sample 3</h4>
<p dir="auto">The length-3 case.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="In [1]: import pandas as pd

In [2]: x = pd.DataFrame()
   ...: x['i'] = [1, 2, 3]
   ...: x['j'] = [11, 22, 33]
   ...: x['k'] = [4, 5, 6]
   ...: x.set_index(['i', 'j'], inplace=True)
   ...: v = x.k
   ...: 

In [3]: v
Out[3]: 
i  j 
1  11    4
2  22    5
3  33    6
Name: k, dtype: int64

In [4]: v.values
Out[4]: array([4, 5, 6])

In [5]: v.iloc[0] += 4; v
Out[5]: 
i  j 
1  11    8
2  22    5
3  33    6
Name: k, dtype: int64

In [6]: # Expected (3, 33) to be incremented but instead (2, 22) is?
   ...: v.iloc[2] += 7; v
   ...: 
Out[6]: 
i  j 
1  11     8
2  22    13
3  33     6
Name: k, dtype: int64

In [7]: # Expected (2, 22) to be incremented but instead (1, 11) is set to (2, 22) + increment?
   ...: v.iloc[1] += 10; v
   ...: 
Out[7]: 
i  j 
1  11    23
2  22    13
3  33     6
Name: k, dtype: int64"><pre class="notranslate"><span class="pl-v">In</span> [<span class="pl-c1">1</span>]: <span class="pl-s1">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>

<span class="pl-v">In</span> [<span class="pl-c1">2</span>]: <span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>()
   ...: <span class="pl-s1">x</span>[<span class="pl-s">'i'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">1</span>, <span class="pl-c1">2</span>, <span class="pl-c1">3</span>]
   ...: <span class="pl-s1">x</span>[<span class="pl-s">'j'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">11</span>, <span class="pl-c1">22</span>, <span class="pl-c1">33</span>]
   ...: <span class="pl-s1">x</span>[<span class="pl-s">'k'</span>] <span class="pl-c1">=</span> [<span class="pl-c1">4</span>, <span class="pl-c1">5</span>, <span class="pl-c1">6</span>]
   ...: <span class="pl-s1">x</span>.<span class="pl-en">set_index</span>([<span class="pl-s">'i'</span>, <span class="pl-s">'j'</span>], <span class="pl-s1">inplace</span><span class="pl-c1">=</span><span class="pl-c1">True</span>)
   ...: <span class="pl-s1">v</span> <span class="pl-c1">=</span> <span class="pl-s1">x</span>.<span class="pl-s1">k</span>
   ...: 

<span class="pl-v">In</span> [<span class="pl-c1">3</span>]: <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">3</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span> 
<span class="pl-c1">1</span>  <span class="pl-c1">11</span>    <span class="pl-c1">4</span>
<span class="pl-c1">2</span>  <span class="pl-c1">22</span>    <span class="pl-c1">5</span>
<span class="pl-c1">3</span>  <span class="pl-c1">33</span>    <span class="pl-c1">6</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">4</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">values</span>
<span class="pl-v">Out</span>[<span class="pl-c1">4</span>]: <span class="pl-en">array</span>([<span class="pl-c1">4</span>, <span class="pl-c1">5</span>, <span class="pl-c1">6</span>])

<span class="pl-v">In</span> [<span class="pl-c1">5</span>]: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">0</span>] <span class="pl-c1">+=</span> <span class="pl-c1">4</span>; <span class="pl-s1">v</span>
<span class="pl-v">Out</span>[<span class="pl-c1">5</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span> 
<span class="pl-c1">1</span>  <span class="pl-c1">11</span>    <span class="pl-c1">8</span>
<span class="pl-c1">2</span>  <span class="pl-c1">22</span>    <span class="pl-c1">5</span>
<span class="pl-c1">3</span>  <span class="pl-c1">33</span>    <span class="pl-c1">6</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">6</span>]: <span class="pl-c"># Expected (3, 33) to be incremented but instead (2, 22) is?</span>
   ...: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">2</span>] <span class="pl-c1">+=</span> <span class="pl-c1">7</span>; <span class="pl-s1">v</span>
   ...: 
<span class="pl-v">Out</span>[<span class="pl-c1">6</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span> 
<span class="pl-c1">1</span>  <span class="pl-c1">11</span>     <span class="pl-c1">8</span>
<span class="pl-c1">2</span>  <span class="pl-c1">22</span>    <span class="pl-c1">13</span>
<span class="pl-c1">3</span>  <span class="pl-c1">33</span>     <span class="pl-c1">6</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span>

<span class="pl-v">In</span> [<span class="pl-c1">7</span>]: <span class="pl-c"># Expected (2, 22) to be incremented but instead (1, 11) is set to (2, 22) + increment?</span>
   ...: <span class="pl-s1">v</span>.<span class="pl-s1">iloc</span>[<span class="pl-c1">1</span>] <span class="pl-c1">+=</span> <span class="pl-c1">10</span>; <span class="pl-s1">v</span>
   ...: 
<span class="pl-v">Out</span>[<span class="pl-c1">7</span>]: 
<span class="pl-s1">i</span>  <span class="pl-s1">j</span> 
<span class="pl-c1">1</span>  <span class="pl-c1">11</span>    <span class="pl-c1">23</span>
<span class="pl-c1">2</span>  <span class="pl-c1">22</span>    <span class="pl-c1">13</span>
<span class="pl-c1">3</span>  <span class="pl-c1">33</span>     <span class="pl-c1">6</span>
<span class="pl-v">Name</span>: <span class="pl-s1">k</span>, <span class="pl-s1">dtype</span>: <span class="pl-s1">int64</span></pre></div>
<h4 dir="auto">Code Sample 3 Problem / Expected Result</h4>
<p dir="auto">Comments inline above</p>
<h4 dir="auto">Output of <code class="notranslate">pd.show_versions()</code></h4>
<details>
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.1.35-pv-ts2
machine: x86_64
processor: 
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.utf8
LOCALE: en_US.UTF-8
<p dir="auto">pandas: 0.20.3<br>
pytest: None<br>
pip: 9.0.1<br>
setuptools: 27.2.0<br>
Cython: None<br>
numpy: 1.13.1<br>
scipy: None<br>
xarray: None<br>
IPython: 6.1.0<br>
sphinx: None<br>
patsy: None<br>
dateutil: 2.6.1<br>
pytz: 2017.2<br>
blosc: None<br>
bottleneck: None<br>
tables: None<br>
numexpr: None<br>
feather: None<br>
matplotlib: None<br>
openpyxl: None<br>
xlrd: None<br>
xlwt: None<br>
xlsxwriter: None<br>
lxml: None<br>
bs4: None<br>
html5lib: None<br>
sqlalchemy: None<br>
pymysql: None<br>
psycopg2: None<br>
jinja2: 2.9.6<br>
s3fs: None<br>
pandas_gbq: None<br>
pandas_datareader: None</p>
</details>
<p dir="auto">I'll propose a</p>
<p dir="auto"><code class="notranslate">cache_datetime=False</code> keyword as an addition to <code class="notranslate">read_csv</code> and <code class="notranslate">pd.to_datetime</code></p>
<p dir="auto">this would use a lookup cache (a dict will probably work), to map datetime strings to Timestamp objects. For repeated dates this will lead to some dramatic speedups.</p>
<p dir="auto">Care must be taken if a <code class="notranslate">format</code> kw is provided (in <code class="notranslate">to_datetime</code> as the cache will have to be exposed). This would be an optional (and default <code class="notranslate">False</code>) as I think if you have unique dates this could modestly slow down things (but can be revisted if needed).</p>
<p dir="auto">This <em>might</em> need also want to accept a list of column names (like <code class="notranslate">parse_dates</code>) to enable per-column caching (e.g. you might want to apply to a column, but not the index of example).</p>
<p dir="auto">possibly we could overload <code class="notranslate">parse_dates='cache'</code> to mean this as well</p>
<p dir="auto">trivial example</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [1]: pd.DataFrame({'A' : ['20130101 00:00:00']*10000}).to_csv('test.csv',index=True)

In [14]: def parser(x):
   ....:         uniques = pd.Series(pd.unique(x))
   ....:         d = pd.to_datetime(uniques)
   ....:         d.index = uniques
   ....:         return Series(x).map(d).values
   ....: 
In [3]: df1 = pd.read_csv('test.csv',index_col=0, parse_dates=['A'])

In [4]: df2 = pd.read_csv('test.csv',index_col=0, parse_dates=['A'], date_parser=parser)

In [17]: %timeit pd.read_csv('test.csv',index_col=0, parse_dates=['A'])
1 loops, best of 3: 969 ms per loop

In [18]: %timeit pd.read_csv('test.csv',index_col=0, parse_dates=['A'], date_parser=parser)
100 loops, best of 3: 5.31 ms per loop

In [7]: df1.equals(df2)
Out[7]: True"><pre class="notranslate"><code class="notranslate">In [1]: pd.DataFrame({'A' : ['20130101 00:00:00']*10000}).to_csv('test.csv',index=True)

In [14]: def parser(x):
   ....:         uniques = pd.Series(pd.unique(x))
   ....:         d = pd.to_datetime(uniques)
   ....:         d.index = uniques
   ....:         return Series(x).map(d).values
   ....: 
In [3]: df1 = pd.read_csv('test.csv',index_col=0, parse_dates=['A'])

In [4]: df2 = pd.read_csv('test.csv',index_col=0, parse_dates=['A'], date_parser=parser)

In [17]: %timeit pd.read_csv('test.csv',index_col=0, parse_dates=['A'])
1 loops, best of 3: 969 ms per loop

In [18]: %timeit pd.read_csv('test.csv',index_col=0, parse_dates=['A'], date_parser=parser)
100 loops, best of 3: 5.31 ms per loop

In [7]: df1.equals(df2)
Out[7]: True
</code></pre></div>