<p dir="auto">{Python 2.6.6, pandas 0.12}</p>
<p dir="auto">A DataFrame will fail to unstack() when one of the columns retained as an index has NaN values.  The code below sets up a dataframe with NaN in some index entries, at which point calling unstack() will fail.</p>
<p dir="auto">In the first failure, the exception message is that the index "has duplicate entries" which is patently false.   In the second failure, where a given id only has one NaN, the error message becomes cannot convert float NaN to integer.</p>
<p dir="auto">A final try, with NaN converted to a sentinel value of 42, shows proper behavior.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import pandas
from numpy import nan

df = pandas.DataFrame(
    {'agent': {
                      17263: 'Hg',
                      17264: 'U',
                      17265: 'Pb',
                      17266: 'Sn',
                      17267: 'Ag',
                      17268: 'Hg'},
    'change': {
                      17263: nan,
                      17264: 0.0,
                      17265: 7.070e-06,
                      17266: 2.3614e-05,
                      17267: 0.0,
                      17268: -0.00015},
    'dosage': {
                      17263: nan,
                      17264: nan,
                      17265: nan,
                      17266: 0.0133,
                      17267: 0.0133,
                      17268: 0.0133},
    's_id': {
                      17263: 680585148,
                      17264: 680585148,
                      17265: 680585148,
                      17266: 680607017,
                      17267: 680607017,
                      17268: 680607017}}
            )
try:
    dupe = df.copy().set_index(['s_id','dosage','agent'])
    badDupe = dupe.unstack()
except Exception as e:
    print( 'Error with all data was: %s'%(e,) )
try:
    getnan = df.ix[17264:].copy().set_index(['s_id','dosage','agent'])
    badNan = getnan.unstack()
except Exception as e:
    print( 'Error dropping first entry was: %s'%(e,) )
df.dosage[:3]=42
willWork = df.copy().set_index(['s_id','dosage','agent'])
u = willWork.unstack()
print(u) "><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">pandas</span>
<span class="pl-k">from</span> <span class="pl-s1">numpy</span> <span class="pl-k">import</span> <span class="pl-s1">nan</span>

<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pandas</span>.<span class="pl-v">DataFrame</span>(
    {<span class="pl-s">'agent'</span>: {
                      <span class="pl-c1">17263</span>: <span class="pl-s">'Hg'</span>,
                      <span class="pl-c1">17264</span>: <span class="pl-s">'U'</span>,
                      <span class="pl-c1">17265</span>: <span class="pl-s">'Pb'</span>,
                      <span class="pl-c1">17266</span>: <span class="pl-s">'Sn'</span>,
                      <span class="pl-c1">17267</span>: <span class="pl-s">'Ag'</span>,
                      <span class="pl-c1">17268</span>: <span class="pl-s">'Hg'</span>},
    <span class="pl-s">'change'</span>: {
                      <span class="pl-c1">17263</span>: <span class="pl-s1">nan</span>,
                      <span class="pl-c1">17264</span>: <span class="pl-c1">0.0</span>,
                      <span class="pl-c1">17265</span>: <span class="pl-c1">7.070e-06</span>,
                      <span class="pl-c1">17266</span>: <span class="pl-c1">2.3614e-05</span>,
                      <span class="pl-c1">17267</span>: <span class="pl-c1">0.0</span>,
                      <span class="pl-c1">17268</span>: <span class="pl-c1">-</span><span class="pl-c1">0.00015</span>},
    <span class="pl-s">'dosage'</span>: {
                      <span class="pl-c1">17263</span>: <span class="pl-s1">nan</span>,
                      <span class="pl-c1">17264</span>: <span class="pl-s1">nan</span>,
                      <span class="pl-c1">17265</span>: <span class="pl-s1">nan</span>,
                      <span class="pl-c1">17266</span>: <span class="pl-c1">0.0133</span>,
                      <span class="pl-c1">17267</span>: <span class="pl-c1">0.0133</span>,
                      <span class="pl-c1">17268</span>: <span class="pl-c1">0.0133</span>},
    <span class="pl-s">'s_id'</span>: {
                      <span class="pl-c1">17263</span>: <span class="pl-c1">680585148</span>,
                      <span class="pl-c1">17264</span>: <span class="pl-c1">680585148</span>,
                      <span class="pl-c1">17265</span>: <span class="pl-c1">680585148</span>,
                      <span class="pl-c1">17266</span>: <span class="pl-c1">680607017</span>,
                      <span class="pl-c1">17267</span>: <span class="pl-c1">680607017</span>,
                      <span class="pl-c1">17268</span>: <span class="pl-c1">680607017</span>}}
            )
<span class="pl-k">try</span>:
    <span class="pl-s1">dupe</span> <span class="pl-c1">=</span> <span class="pl-s1">df</span>.<span class="pl-en">copy</span>().<span class="pl-en">set_index</span>([<span class="pl-s">'s_id'</span>,<span class="pl-s">'dosage'</span>,<span class="pl-s">'agent'</span>])
    <span class="pl-s1">badDupe</span> <span class="pl-c1">=</span> <span class="pl-s1">dupe</span>.<span class="pl-en">unstack</span>()
<span class="pl-k">except</span> <span class="pl-v">Exception</span> <span class="pl-k">as</span> <span class="pl-s1">e</span>:
    <span class="pl-en">print</span>( <span class="pl-s">'Error with all data was: %s'</span><span class="pl-c1">%</span>(<span class="pl-s1">e</span>,) )
<span class="pl-k">try</span>:
    <span class="pl-s1">getnan</span> <span class="pl-c1">=</span> <span class="pl-s1">df</span>.<span class="pl-s1">ix</span>[<span class="pl-c1">17264</span>:].<span class="pl-en">copy</span>().<span class="pl-en">set_index</span>([<span class="pl-s">'s_id'</span>,<span class="pl-s">'dosage'</span>,<span class="pl-s">'agent'</span>])
    <span class="pl-s1">badNan</span> <span class="pl-c1">=</span> <span class="pl-s1">getnan</span>.<span class="pl-en">unstack</span>()
<span class="pl-k">except</span> <span class="pl-v">Exception</span> <span class="pl-k">as</span> <span class="pl-s1">e</span>:
    <span class="pl-en">print</span>( <span class="pl-s">'Error dropping first entry was: %s'</span><span class="pl-c1">%</span>(<span class="pl-s1">e</span>,) )
<span class="pl-s1">df</span>.<span class="pl-s1">dosage</span>[:<span class="pl-c1">3</span>]<span class="pl-c1">=</span><span class="pl-c1">42</span>
<span class="pl-s1">willWork</span> <span class="pl-c1">=</span> <span class="pl-s1">df</span>.<span class="pl-en">copy</span>().<span class="pl-en">set_index</span>([<span class="pl-s">'s_id'</span>,<span class="pl-s">'dosage'</span>,<span class="pl-s">'agent'</span>])
<span class="pl-s1">u</span> <span class="pl-c1">=</span> <span class="pl-s1">willWork</span>.<span class="pl-en">unstack</span>()
<span class="pl-en">print</span>(<span class="pl-s1">u</span>) </pre></div>
<p dir="auto">Overall output:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Error with all data was: Index contains duplicate entries, cannot reshape
Error dropping first entry was: cannot convert float NaN to integer

                   change                                 
agent                  Ag       Hg        Pb        Sn   U
s_id      dosage                                          
680585148 42.0000     NaN      NaN  0.000007       NaN   0
680607017 0.0133        0 -0.00015       NaN  0.000024 NaN"><pre class="notranslate"><code class="notranslate">Error with all data was: Index contains duplicate entries, cannot reshape
Error dropping first entry was: cannot convert float NaN to integer

                   change                                 
agent                  Ag       Hg        Pb        Sn   U
s_id      dosage                                          
680585148 42.0000     NaN      NaN  0.000007       NaN   0
680607017 0.0133        0 -0.00015       NaN  0.000024 NaN
</code></pre></div>
<h4 dir="auto">Add the following function to pd. dataFrame and pd.Series</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="def ends(df, x=5):
    &quot;&quot;&quot;Returns both head and tail of the dataframe or series.
    
    Args:
        x (int): Optional number of rows to return for each head and tail
    &quot;&quot;&quot;
    print('{} rows x {} columns'.format(np.shape(df)[0],np.shape(df)[1]))
    return df.head(x).append(df.tail(x))
"><pre class="notranslate"><span class="pl-k">def</span> <span class="pl-en">ends</span>(<span class="pl-s1">df</span>, <span class="pl-s1">x</span><span class="pl-c1">=</span><span class="pl-c1">5</span>):
    <span class="pl-s">"""Returns both head and tail of the dataframe or series.</span>
<span class="pl-s">    </span>
<span class="pl-s">    Args:</span>
<span class="pl-s">        x (int): Optional number of rows to return for each head and tail</span>
<span class="pl-s">    """</span>
    <span class="pl-en">print</span>(<span class="pl-s">'{} rows x {} columns'</span>.<span class="pl-en">format</span>(<span class="pl-s1">np</span>.<span class="pl-en">shape</span>(<span class="pl-s1">df</span>)[<span class="pl-c1">0</span>],<span class="pl-s1">np</span>.<span class="pl-en">shape</span>(<span class="pl-s1">df</span>)[<span class="pl-c1">1</span>]))
    <span class="pl-k">return</span> <span class="pl-s1">df</span>.<span class="pl-en">head</span>(<span class="pl-s1">x</span>).<span class="pl-en">append</span>(<span class="pl-s1">df</span>.<span class="pl-en">tail</span>(<span class="pl-s1">x</span>))</pre></div>
<h4 dir="auto">Problem description</h4>
<p dir="auto">Often both the beginning and end of a df are of interest, fore example in a time series.</p>
<p dir="auto">This leads to calling df.head() df.tail()  in two seperate notebook cells. This is not only tedious, but also leads to a cluttered notebook. A function that returns both of these + a print on the number of rows and columns, thus allowing a check if the index matches the number of rows.</p>
<h4 dir="auto">Example</h4>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="
import pandas as pd
import numpy as np
df = pd.DataFrame(np.random.rand(1500,6))
print(ends(df,2))"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">pandas</span> <span class="pl-k">as</span> <span class="pl-s1">pd</span>
<span class="pl-k">import</span> <span class="pl-s1">numpy</span> <span class="pl-k">as</span> <span class="pl-s1">np</span>
<span class="pl-s1">df</span> <span class="pl-c1">=</span> <span class="pl-s1">pd</span>.<span class="pl-v">DataFrame</span>(<span class="pl-s1">np</span>.<span class="pl-s1">random</span>.<span class="pl-en">rand</span>(<span class="pl-c1">1500</span>,<span class="pl-c1">6</span>))
<span class="pl-en">print</span>(<span class="pl-en">ends</span>(<span class="pl-s1">df</span>,<span class="pl-c1">2</span>))</pre></div>
<p dir="auto">1500 rows x 6 columns</p>
<table role="table">
<thead>
<tr>
<th></th>
<th>0</th>
<th>1</th>
<th>2</th>
<th>3</th>
<th>4</th>
<th>5</th>
</tr>
</thead>
<tbody>
<tr>
<td>0</td>
<td>0.949695</td>
<td>0.160928</td>
<td>0.434134</td>
<td>0.943103</td>
<td>0.477830</td>
<td>0.903479</td>
</tr>
<tr>
<td>1</td>
<td>0.736711</td>
<td>0.103746</td>
<td>0.028694</td>
<td>0.205910</td>
<td>0.226061</td>
<td>0.458452</td>
</tr>
<tr>
<td>1498</td>
<td>0.362950</td>
<td>0.586887</td>
<td>0.399681</td>
<td>0.115366</td>
<td>0.239049</td>
<td>0.386281</td>
</tr>
<tr>
<td>1499</td>
<td>0.018102</td>
<td>0.852198</td>
<td>0.880993</td>
<td>0.671604</td>
<td>0.705586</td>
<td>0.802237</td>
</tr>
</tbody>
</table>
<h4 dir="auto">Output of <code class="notranslate">pd.show_versions()</code></h4>
<details>
<h2 dir="auto">INSTALLED VERSIONS</h2>
<p dir="auto">commit: None<br>
python: 3.6.2.final.0<br>
python-bits: 64<br>
OS: Windows<br>
OS-release: 8.1<br>
machine: AMD64<br>
processor: Intel64 Family 6 Model 61 Stepping 4, GenuineIntel<br>
byteorder: little<br>
LC_ALL: None<br>
LANG: None<br>
LOCALE: None.None</p>
<p dir="auto">pandas: 0.21.0<br>
pytest: 3.3.1<br>
pip: 9.0.1<br>
setuptools: 38.2.4<br>
Cython: 0.26.1<br>
numpy: 1.12.1<br>
scipy: 0.19.1<br>
pyarrow: None<br>
xarray: None<br>
IPython: 6.1.0<br>
sphinx: 1.6.3<br>
patsy: 0.4.1<br>
dateutil: 2.6.1<br>
pytz: 2017.3<br>
blosc: None<br>
bottleneck: 1.2.1<br>
tables: 3.4.2<br>
numexpr: 2.6.2<br>
feather: None<br>
matplotlib: 2.0.2<br>
openpyxl: 2.4.8<br>
xlrd: 1.1.0<br>
xlwt: 1.3.0<br>
xlsxwriter: 0.9.8<br>
lxml: 3.8.0<br>
bs4: 4.6.0<br>
html5lib: 0.999999999<br>
sqlalchemy: 1.1.13<br>
pymysql: None<br>
psycopg2: None<br>
jinja2: 2.10<br>
s3fs: None<br>
fastparquet: None<br>
pandas_gbq: None<br>
pandas_datareader: None</p>
</details>