title: Optimized Algorithm for appending ghost row to an existing dataframe in Python
body: <p>I have a dataframe and I want to append ghost rows ( copy of existing row) to the dataframe.</p>
  
  <pre><code>       id   month  as_of_date1 turn  age 
  119 5712    201401  2014-01-01  9   0
  120 5712    201402  2014-02-01  9   1
  121 5712    201403  2014-03-01  9   2
  122 5712    201404  2014-04-01  9   3
  123 5712    201405  2014-05-01  9   4
  124 5712    201406  2014-06-01  9   5
  125 9130    201401  2014-01-01  9   0
  126 9130    201402  2014-02-01  9   1
  127 9130    201403  2014-03-01  9   2
  128 9130    201404  2014-04-01  9   3
  129 9130    201405  2014-05-01  9   4
  </code></pre>
  
  <p>The ghost rows are selected by conditions:
  if age is less than turn we need to append the latest row till <code>age== turn of</code> or <code>as_of_date1 == now()</code></p>
  
  <p>right now I'm using the following code but since the data is large around 200k rows with 100 fields it takes for ever</p>
  
  <pre><code>tdf1=tdf.loc[(tdf['age']&lt;tdf['turn'])]
  tdf2=tdf1.drop_duplicates(subset=['id'],keep='last')
  leads=tdf2.index.tolist()
  for lead in leads:
      ttdf=tdf.loc[[lead]]
      diff1 = relativedelta.relativedelta(datetime.datetime(2018,6,1),tdf.loc[lead,'as_of_date1']).months
      diff2=tdf.loc[lead,'turn']-tdf.loc[lead,'age']
      diff=min(diff1,diff2)
      for i in range(0,diff):
          tdf = tdf.append(ttdf, ignore_index=True)
  </code></pre>
  
  <p>Expected outcome:</p>
  
  <pre><code>    id   month  as_of_date1 turn  age 
  119 5712    201401  2014-01-01  9   0
  120 5712    201402  2014-02-01  9   1
  121 5712    201403  2014-03-01  9   2
  122 5712    201404  2014-04-01  9   3
  123 5712    201405  2014-05-01  9   4
  124 5712    201406  2014-06-01  9   5
  125 9130    201401  2014-01-01  9   0
  126 9130    201402  2014-02-01  9   1
  127 9130    201403  2014-03-01  9   2
  128 9130    201404  2014-04-01  9   3
  129 9130    201405  2014-05-01  9   4
  130 5712    201406  2014-06-01  9   5
  131 5712    201406  2014-06-01  9   5
  132 5712    201406  2014-06-01  9   5
  133 5712    201406  2014-06-01  9   5
  134 9130    201405  2014-05-01  9   4
  135 9130    201405  2014-05-01  9   4
  136 9130    201405  2014-05-01  9   4
  137 9130    201405  2014-05-01  9   4
  138 9130    201405  2014-05-01  9   4
  </code></pre>
  
  <p>I would appreciate if anyone knows a faster algorithm</p>