title: Pandas/pyspark representative sample with occurences in different column
body: <p>I'm looking for creating a representative sample of specific dataframe.</p>
  <p>My df is looks like:</p>
  <pre><code>query    | total
  ---------|-------
  facebook | 123456
  monkey   | 3456
  iphone   | 54321
  laptop   | 1234
  headset  | 3333
  plates   | 4333
  girl     | 11222
  .
  .
  .
  </code></pre>
  <p>I understood representative as:</p>
  <ol>
  <li>every query can be displayed max once</li>
  <li>every query has a probability of displaying by it's occurrence (in my case <code>total</code> column)</li>
  </ol>
  <p>If I create a huge list (dataset with 1 col) with 123456 occurences of <code>facebook</code>, 3456 occurences of <code>monkey</code> etc and then do df.sample(something, something) it might work but I see it as super unefficient (considering having a huge dataset - literally millions, maybe billions of lines after putting into the &quot;list&quot; in my case)</p>
  <p>Is here any other different (much more efficient) way of getting representative sample? Can be done in pandas or pyspark.</p>
  <p>Example:</p>
  <p>df with 8 queries and 20 occurences.</p>
  <pre><code>q1 | 1
  q2 | 1
  q3 | 5
  q4 | 1
  q5 | 2
  q6 | 4
  q7 | 4
  q8 | 2
  </code></pre>
  <p>Let's say I need representative sample of 5 queries. Every query has a probability of being picked. q1 has 5%, q2 has 5%, q3 has 25% etc. Let's divide it into rolls. After the 1st roll - only one query is picked, for example q3. q3 is added into the final output and we roll again. 2. roll q5 is being picked and added into final output. 3. roll q3 is being picked AGAIN but it is not added into the final output bcs it is already there. Then we roll again and again after 5 spots of final output is being filled.</p>
  <p><strong>Point is that a query with more occurences has higher chance being in the final output.</strong></p>
  <p>Sadly I have much bigger datasets and it's not really possible for me to pick queries like this bcs the list of items (123456x facebook, 3456x monkey) would be way too big.</p>