title: spark.default.parallelism for Parallelize RDD defaults to 2 for spark submit
body: <p>Spark standalone cluster with a master and 2 worker nodes 4 cpu core on each worker. Total 8 cores for all workers.</p>
  
  <p>When running the following via spark-submit (spark.default.parallelism is not set)</p>
  
  <pre class="lang-scala prettyprint-override"><code>val myRDD = sc.parallelize(1 to 100000)
  println("Partititon size - " + myRDD.partitions.size)
  val totl = myRDD.reduce((x, y) =&gt; x + y)
  println("Sum - " + totl)
  </code></pre>
  
  <p>It returns value 2 for partition size.</p>
  
  <p>When using spark-shell  by connecting to spark standalone cluster the same code returns correct partition size 8.  </p>
  
  <p>What can be the reason ?</p>
  
  <p>Thanks. </p>