title: Run Scala Spark with SBT
body: <p>The code below causes Spark to become unresponsive: </p>
  
  <pre><code>System.setProperty("hadoop.home.dir", "H:\\winutils");
  
  val sparkConf = new SparkConf().setAppName("GroupBy Test").setMaster("local[1]")
  val sc = new SparkContext(sparkConf)
  
  def main(args: Array[String]) {
  
      val text_file = sc.textFile("h:\\data\\details.txt")
  
      val counts = text_file
        .flatMap(line =&gt; line.split(" "))
        .map(word =&gt; (word, 1))
        .reduceByKey(_ + _)
  
      println(counts);
  
  }
  </code></pre>
  
  <p>I'm setting hadoop.home.dir in order to avoid the error mentioned here: <a href="https://stackoverflow.com/questions/19620642/failed-to-locate-the-winutils-binary-in-the-hadoop-binary-path">Failed to locate the winutils binary in the hadoop binary path</a></p>
  
  <p>This is how my build.sbt file looks like: </p>
  
  <pre><code>lazy val root = (project in file(".")).
    settings(
      name := "hello",
      version := "1.0",
      scalaVersion := "2.11.0"
    )
  
  
  libraryDependencies  ++= Seq(
  
              "org.apache.spark" % "spark-core_2.11" % "1.6.0"
  
  )
  </code></pre>
  
  <p>Should Scala Spark be compilable/runnable using the sbt code in the file?</p>
  
  <p>I think code is fine, it was taken verbatim from <a href="http://spark.apache.org/examples.html" rel="nofollow noreferrer">http://spark.apache.org/examples.html</a>, but I am not sure if the Hadoop WinUtils path is required.</p>
  
  <p>Update: "The solution was to use fork := true in the main build.sbt"
  Here is the reference: <a href="https://stackoverflow.com/questions/28612837/spark-classnotfoundexception-when-running-hello-world-example-in-scala-2-11">Spark: ClassNotFoundException when running hello world example in scala 2.11</a></p>