title: Is it possible to add Fair scheduler pools programmatically in Spark?
body: <p>I'm developing an application where several users use the same SparkContext to launch their queries to a Spark Cluster. </p>
  
  <p>As the Spark documentation states (<a href="https://spark.apache.org/docs/2.2.0/job-scheduling.html#fair-scheduler-pools" rel="nofollow noreferrer">https://spark.apache.org/docs/2.2.0/job-scheduling.html#fair-scheduler-pools</a>), with the Fair scheduler, you can assign a different pool for every user and they'll get a fair share of the cluster resources but every pool will be set up with the default pool configuration (scheduling mode FIFO, weight 1, and minShare 0).</p>
  
  <p>Given that we don't know in advance which users can connect to the application, we can't set up a configuration file for the fair scheduler pools for all the users.</p>
  
  <p>So, in order to give a pool to every user dynamically and set up every pool with a FAIR scheduling mode, I think there might be 2 options:</p>
  
  <ol>
  <li>Change the default pool behaviour in order to change the scheduling mode to FAIR. Is it possible? How?</li>
  <li>Generate a scheduler pool dynamically and programmatically in order to add a scheduler pool when a user connects to the application for first time and that pool should be created with a FAIR scheduling mode. Is it possible? How?</li>
  </ol>
  
  <p>Thanks in advance</p>