title: How do I read a Scala serialization stack (from Spark)?
body: <p>How do I read a serialization stack?</p>
  
  <p>I'm building a distributed NLP application on top of Spark. I periodically run into these NotSerializable exceptions, and always muddle my way through them. But, I've never found good documentation on what everything in the Serialization stack means. </p>
  
  <p>How do I read a Serialization stack accompanying a NotSerializable error in Scala? How do I pinpoint the class or object that is causing the error? What is the significance of the 'field', 'object', 'writeObject' and 'writeReplace' fields in the stack? </p>
  
  <p>An example is below:</p>
  
  <pre><code>Caused by: java.io.NotSerializableException: MyPackage.testing.PreprocessTest$$typecreator1$1
  Serialization stack:
          - object not serializable (class: MyPackage.testing.PreprocessTest$$typecreator1$1, value: MyPackage.testing.PreprocessTest$$typecreator1$1@27f6854b)
          - writeObject data (class: scala.reflect.api.SerializedTypeTag)
          - object (class scala.reflect.api.SerializedTypeTag, scala.reflect.api.SerializedTypeTag@4a571516)
          - writeReplace data (class: scala.reflect.api.SerializedTypeTag)
          - object (class scala.reflect.api.TypeTags$TypeTagImpl, TypeTag[String])
          - field (class: MyPackage.package$$anonfun$deserializeMaps$1, name: evidence$1$1, type: interface scala.reflect.api.TypeTags$TypeTag)
          - object (class MyPackage.package$$anonfun$deserializeMaps$1, &lt;function1&gt;)
          - field (class: MyPackage.package$$anonfun$deserializeMaps$1$$anonfun$apply$4, name: $outer, type: class MyPackage.package$$anonfun$deserializeMaps$1)
          - object (class MyPackage.package$$anonfun$deserializeMaps$1$$anonfun$apply$4, &lt;function1&gt;)
          - field (class: MyPackage.package$$anonfun$deserializeMaps$1$$anonfun$apply$4$$anonfun$apply$5, name: $outer, type: class MyPackage.package$$anonfun$deserializeMaps$1$$anonfun$apply$4)
          - object (class MyPackage.package$$anonfun$deserializeMaps$1$$anonfun$apply$4$$anonfun$apply$5, &lt;function1&gt;)
          - field (class: org.apache.spark.sql.catalyst.expressions.ScalaUDF$$anonfun$2, name: func$2, type: interface scala.Function1)
          - object (class org.apache.spark.sql.catalyst.expressions.ScalaUDF$$anonfun$2, &lt;function1&gt;)
          - field (class: org.apache.spark.sql.catalyst.expressions.ScalaUDF, name: f, type: interface scala.Function1)
          - object (class org.apache.spark.sql.catalyst.expressions.ScalaUDF, UDF(UDF(tokenMap#149)))
          - field (class: org.apache.spark.sql.catalyst.expressions.Alias, name: child, type: class org.apache.spark.sql.catalyst.expressions.Expression)
          - object (class org.apache.spark.sql.catalyst.expressions.Alias, UDF(UDF(tokenMap#149)) AS tokenMap#3131)
          - writeObject data (class: scala.collection.immutable.$colon$colon)
          - object (class scala.collection.immutable.$colon$colon, List(id#148, UDF(UDF(tokenMap#149)) AS tokenMap#3131, UDF(UDF(bigramMap#150)) AS bigramMap#3132, sentences#151, se_sentence_count#152, se_word_count#153, se_subjective_count#154, se_objective_count#155, se_document_sentiment#156, UDF(UDF(se_category#157)) AS se_category#3133))
          - field (class: org.apache.spark.sql.execution.Project, name: projectList, type: interface scala.collection.Seq)
          - object (class org.apache.spark.sql.execution.Project, Project [id#148,UDF(UDF(tokenMap#149)) AS tokenMap#3131,UDF(UDF(bigramMap#150)) AS bigramMap#3132,sentences#151,se_sentence_count#152,se_word_count#153,se_subjective_count#154,se_objective_count#155,se_document_sentiment#156,UDF(UDF(se_category#157)) AS se_category#3133]
  +- InMemoryColumnarTableScan [se_sentence_count#152,bigramMap#150,id#148,tokenMap#149,se_word_count#153,sentences#151,se_document_sentiment#156,se_subjective_count#154,se_category#157,se_objective_count#155], InMemoryRelation [id#148,tokenMap#149,bigramMap#150,sentences#151,se_sentence_count#152,se_word_count#153,se_subjective_count#154,se_objective_count#155,se_document_sentiment#156,se_category#157], true, 10000, StorageLevel(true, true, false, true, 1), Union, None
  )
          - field (class: org.apache.spark.sql.execution.ConvertToSafe, name: child, type: class org.apache.spark.sql.execution.SparkPlan)
          - object (class org.apache.spark.sql.execution.ConvertToSafe, ConvertToSafe
  +- Project [id#148,UDF(UDF(tokenMap#149)) AS tokenMap#3131,UDF(UDF(bigramMap#150)) AS bigramMap#3132,sentences#151,se_sentence_count#152,se_word_count#153,se_subjective_count#154,se_objective_count#155,se_document_sentiment#156,UDF(UDF(se_category#157)) AS se_category#3133]
     +- InMemoryColumnarTableScan [se_sentence_count#152,bigramMap#150,id#148,tokenMap#149,se_word_count#153,sentences#151,se_document_sentiment#156,se_subjective_count#154,se_category#157,se_objective_count#155], InMemoryRelation [id#148,tokenMap#149,bigramMap#150,sentences#151,se_sentence_count#152,se_word_count#153,se_subjective_count#154,se_objective_count#155,se_document_sentiment#156,se_category#157], true, 10000, StorageLevel(true, true, false, true, 1), Union, None
  )
          - field (class: org.apache.spark.sql.execution.ConvertToSafe$$anonfun$2, name: $outer, type: class org.apache.spark.sql.execution.ConvertToSafe)
          - object (class org.apache.spark.sql.execution.ConvertToSafe$$anonfun$2, &lt;function1&gt;)
  </code></pre>