title: How to submit multiple spark queries parallel using SqlContext
body: <p>I want to run multiple spark SQL parallel in a spark cluster, so that I can <strong>utilize the complete resource</strong>. I'm using <code>sqlContext.sql(query)</code>.</p>
  
  <p><strong>EDITS</strong></p>
  
  <p>I saw some sample code <a href="https://stackoverflow.com/questions/50056831/how-can-i-parallelize-different-sparksql-execution-efficiently/50058194#50058194">here</a> like follows,</p>
  
  <pre><code>val parallelism = 10
  val executor = Executors.newFixedThreadPool(parallelism)
  val ec: ExecutionContext = ExecutionContext.fromExecutor(executor)
  val tasks: Seq[String] = ???
  val results: Seq[Future[Int]] = tasks.map(query =&gt; {
    Future{
      //spark stuff here
      0
    }(ec)
  })
  val allDone: Future[Seq[Int]] = Future.sequence(results)
  //wait for results
  Await.result(allDone, scala.concurrent.duration.Duration.Inf)
  executor.shutdown //otherwise jvm will probably not exit 
  </code></pre>
  
  <p>As I understood, the <strong>ExecutionContext</strong> compute the available cores in the machine(using <strong>ForkJoinPool</strong>) and do the parallelism accordingly.
  But what happens if we consider the spark cluster other-than the single machine and How can it guarantee the complete cluster resource utilization.?</p>
  
  <p>eg: If I have a 10 node cluster with each 4 cores, then how can the above code guarantees that the 40 cores will be utilized.</p>