title: Appending new column to dask dataframe
body: <p>This is a follow up question to <a href="https://stackoverflow.com/questions/46842155/shuffling-data-in-dask/46848375#46848375">Shuffling data in dask</a>.</p>
  
  <p>I have an existing dask dataframe <code>df</code> where I wish to do the following:</p>
  
  <pre><code>df['rand_index'] = np.random.permutation(len(df))
  </code></pre>
  
  <p>However, this gives the error, <code>Column assignment doesn't support type ndarray</code>. I tried to use <code>df.assign(rand_index = np.random.permutation(len(df))</code> which gives the same error.</p>
  
  <p>Here is a minimal (not) working sample:</p>
  
  <pre><code>import pandas as pd
  import dask.dataframe as dd
  import numpy as np
  
  df = dd.from_pandas(pd.DataFrame({'A':[1,2,3]*10, 'B':[3,2,1]*10}), npartitions=10)
  df['rand_index'] = np.random.permutation(len(df))
  </code></pre>
  
  <h2>Note:</h2>
  
  <p>The previous question mentioned using <code>df = df.map_partitions(add_random_column_to_pandas_dataframe, ...)</code> but I'm not sure if that is relevant to this particular case.</p>
  
  <h2>Edit 1</h2>
  
  <p>I attempted
  <code>df['rand_index'] = dd.from_array(np.random.permutation(len_df))</code> which, executed without an issue. When I inspected <code>df.head()</code> it seems that the new column was created just fine. However, when I look at <code>df.tail()</code> the <code>rand_index</code> is a bunch of <code>NaN</code>s.</p>
  
  <p>In fact just to confirm I checked <code>df.rand_index.max().compute()</code> which turned out to be smaller than <code>len(df)-1</code>. So this is probably where <code>df.map_partitions</code> comes into play as I suspect this is an issue with dask being partitioned. In my particular case I have 80 partitions (not referring to the sample case). </p>