title: mitigating a performance warning from pandas (DataFrame is highly fragmented)
body: <p>Suppose we have a function <code>bar(df)</code> that returns a numpy array of length <code>len(df)</code> given a dataframe <code>df</code>.</p>
  <p>Now consider the idiom</p>
  <pre><code>def foo(df):
      for i in range(N):
          df['FOO_' + str(i)] = bar(df)
      return df
  </code></pre>
  <p>A recent pandas update started to cause the following warning</p>
  <blockquote>
  <p>PerformanceWarning: DataFrame is highly fragmented.  This is usually
  the result of calling <code>frame.insert</code> many times, which has poor
  performance.  Consider joining all columns at once using
  pd.concat(axis=1) instead.  To get a de-fragmented frame, use
  <code>newframe = frame.copy()</code></p>
  </blockquote>
  <p>As far as I understand, a way to mitigate this is to change the above code to the following idiom</p>
  <pre><code>def foo2(df):
      frames = [df]
      for i in range(N):
          frames += [pd.Series(bar(df), index=df.index)]
      return pd.concat(frames, axis=1)
  </code></pre>
  <p>The above code fixes the warning, but results in much worse execution times.</p>
  <pre><code>In [110]: %timeit foo()
  1.73 s ± 11 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
  
  In [111]: %timeit foo2()
  2.51 s ± 25.1 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
  </code></pre>
  <p>A fix to suppress a performance warning that introduces extra overhead seems silly. Therefore my question is</p>
  <blockquote>
  <p>How can I fix the warning while also obtaning better performance. In other words, is there a way to improve the function foo2 to offer better performance than foo?</p>
  </blockquote>