title: Spark ClassNotFoundException with the simplest rdd.map
body: <p><strong>edit</strong>
  This question is not a duplicate of <a href="https://stackoverflow.com/questions/41383460/resolving-dependency-problems-in-apache-spark">Resolving dependency problems in Apache Spark</a>:</p>
  
  <ul>
  <li>the answers above are generalities about sharing code. I don't see how it applies to my issue</li>
  <li>Here I do not use spark-submit but simply call spark from an app running on my computer</li>
  <li>There is no dependency whatsoever. The only code to share is the simplistic function <code>i =&gt; i</code></li>
  <li>if it is a problem of spark/java/scala version, I don't understand why it would produce a ClassNotFoundException for the function in question</li>
  </ul>
  
  <hr>
  
  <p>I run this code</p>
  
  <pre><code>object Tmp {
  
    def main(args: Array[String]): Unit = {
      val sparkConf = new SparkConf()
        .setAppName("Spark Test")
        .setMaster("spark://[ip-of-spark-master]:7077")
      val sc = new SparkContext(sparkConf)
  
      sc.parallelize(0 until 10000).map(i =&gt; i).count()
    }
  }
  </code></pre>
  
  <p>Which fails with <code>java.lang.ClassNotFoundException: Tmp$$anonfun$main$1</code> when I simply run it on my computer, without sending any jar. It works if I don't do the <code>map(i =&gt; i)</code> (i.e. it is the function <code>i =&gt; i</code> that is not found).</p>
  
  <p>To be clear: </p>
  
  <ul>
  <li>I do not use spark-submit</li>
  <li>my computer is the driver and is not on the spark cluster</li>
  <li>I use the exact same spark version on my computer that is used in the cluster (2.0.2)</li>
  </ul>
  
  <p>Now I thought that for simple function, spark would serialize it and there were no need to send the jar. I am pretty sure I have done it before.</p>
  
  <p>Did I miss something ?</p>
  
  <p>Looking at the stack trace, it fails on the driver, so on my side:</p>
  
  <pre><code>Driver stacktrace:
  [error]     at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1454)
  [error]     at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1442)
  [error]     at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1441)
  
  ...
  
  [error] Caused by: java.lang.ClassNotFoundException: Tmp$$anonfun$1
  [error]     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  [error]     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  [error]     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  [error]     at java.lang.Class.forName0(Native Method)
  [error]     at java.lang.Class.forName(Class.java:348)
  [error]     at org.apache.spark.serializer.JavaDeserializationStream$$anon$1.resolveClass(JavaSerializer.scala:67)
  ...
  </code></pre>