title: How can i speed up data labelling for a large pandas dataframe?
body: <p>I have a large pandas data frame which roughly looks this</p>
  <pre><code>  Identity  periods      one        two       three     Label
  0   one      1       -0.462407    0.022811  -0.277357
  1   one      1       -0.617588    1.667191  -0.370436
  2   one      2       -0.604699    0.635473  -0.556088
  3   one      2       -0.852943    1.087415  -0.784377
  4   two      3        0.421453    2.390097   0.176333
  5   two      3       -0.447321   -1.215280  -0.187156
  6   two      4        0.398953   -0.334095  -1.194132
  7   two      4       -0.324348   -0.842357   0.970825
  </code></pre>
  <p>I need to be able to categorise the data according to groupings in the various columns, for example one of my categorisation criteria is to label each of the groups in the identity column with a label if there is between x and y periods in the periods column.</p>
  <p>The code I have to categorise this looks like this, generating a final column:</p>
  <pre><code>for i in df['Identity'].unique():
      if (2 &lt;= df[df['Identity']==i]['periods'].max() &lt;= 5) :
          df.loc[df['Identity']==i,'label']='label 1'
  </code></pre>
  <p>I have also tried a version using</p>
  <p><code>df.groupby('Identity').apply().</code></p>
  <p>But this is no quicker.</p>
  <p>My data is approximately 2.8m rows at the moment, and there are about 900 unique identities. The code takes about 5 minutes to run, which to me suggests it's the code within the loop that is slow, rather than the looping making it slow.</p>