title: Apache storm - java.lang.NoClassDefFoundError: com/google/gson/Gson
body: <p>I'm using Apache Storm 0.9.4.
  I set up a 5 nodes cluster and it works fine.
  (Actually 5 supervisors are working as docker containers on 5 different physical nodes.)</p>
  
  <p>My environment is here</p>
  
  <hr>
  
  <pre><code>$cat /etc/redhat-release
  CentOS release 6.6 (Final)
  </code></pre>
  
  <hr>
  
  <pre><code>$docker -v
  Docker version 1.4.1, build 5bc2ff8/1.4.1
  </code></pre>
  
  <hr>
  
  <pre><code>$java -version
  java version "1.7.0_79" OpenJDK Runtime Environment
  (rhel-2.5.5.1.el6_6-x86_64 u79-b14) OpenJDK 64-Bit Server VM (build
  24.79-b02, mixed mode)
  </code></pre>
  
  <hr>
  
  <p>Next, I create a maven project to submit a topology using Netbeans.</p>
  
  <p>There are 1 topology and 1 spout.(very simple)</p>
  
  <ul>
  <li>MyTopology.java</li>
  <li>LoggerSpout.java</li>
  </ul>
  
  <p>MyTopology works fine without GSON.
  But, after I added <code>Gson gson = new Gson();</code> in the <code>nextTuple()</code> function,
  I got java.lang.NoClassDefFoundError: com/google/gson/Gson.</p>
  
  <p>Here is what I did
   -
   1. <code>mvn clean</code> -> <code>mvn compile</code> -> <code>mvn package</code> -> BUILD SUCCESSFULY
   2. <code>storm jar ~/Desktop/teststorm/target/teststorm-1.0.jar jp.soushi.teststorm.MyTopology</code> -> Submitting is OK.</p>
  
  <pre><code>Running: /Library/Java/JavaVirtualMachines/jdk1.7.0_80.jdk/Contents/Home/bin/java -client -Dstorm.options= -Dstorm.home=/usr/local/apache-storm-0.9.4 -Dstorm.log.dir=/usr/local/apache-storm-0.9.4/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/local/apache-storm-0.9.4/lib/asm-4.0.jar:/usr/local/apache-storm-0.9.4/lib/carbonite-1.4.0.jar:/usr/local/apache-storm-0.9.4/lib/chill-java-0.3.5.jar:/usr/local/apache-storm-0.9.4/lib/clj-stacktrace-0.2.2.jar:/usr/local/apache-storm-0.9.4/lib/clj-time-0.4.1.jar:/usr/local/apache-storm-0.9.4/lib/clojure-1.5.1.jar:/usr/local/apache-storm-0.9.4/lib/clout-1.0.1.jar:/usr/local/apache-storm-0.9.4/lib/commons-codec-1.6.jar:/usr/local/apache-storm-0.9.4/lib/commons-exec-1.1.jar:/usr/local/apache-storm-0.9.4/lib/commons-fileupload-1.2.1.jar:/usr/local/apache-storm-0.9.4/lib/commons-io-2.4.jar:/usr/local/apache-storm-0.9.4/lib/commons-lang-2.5.jar:/usr/local/apache-storm-0.9.4/lib/commons-logging-1.1.3.jar:/usr/local/apache-storm-0.9.4/lib/compojure-1.1.3.jar:/usr/local/apache-storm-0.9.4/lib/core.incubator-0.1.0.jar:/usr/local/apache-storm-0.9.4/lib/disruptor-2.10.1.jar:/usr/local/apache-storm-0.9.4/lib/hiccup-0.3.6.jar:/usr/local/apache-storm-0.9.4/lib/jetty-6.1.26.jar:/usr/local/apache-storm-0.9.4/lib/jetty-util-6.1.26.jar:/usr/local/apache-storm-0.9.4/lib/jgrapht-core-0.9.0.jar:/usr/local/apache-storm-0.9.4/lib/jline-2.11.jar:/usr/local/apache-storm-0.9.4/lib/joda-time-2.0.jar:/usr/local/apache-storm-0.9.4/lib/json-simple-1.1.jar:/usr/local/apache-storm-0.9.4/lib/kryo-2.21.jar:/usr/local/apache-storm-0.9.4/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/apache-storm-0.9.4/lib/logback-classic-1.0.13.jar:/usr/local/apache-storm-0.9.4/lib/logback-core-1.0.13.jar:/usr/local/apache-storm-0.9.4/lib/math.numeric-tower-0.0.1.jar:/usr/local/apache-storm-0.9.4/lib/minlog-1.2.jar:/usr/local/apache-storm-0.9.4/lib/objenesis-1.2.jar:/usr/local/apache-storm-0.9.4/lib/reflectasm-1.07-shaded.jar:/usr/local/apache-storm-0.9.4/lib/ring-core-1.1.5.jar:/usr/local/apache-storm-0.9.4/lib/ring-devel-0.3.11.jar:/usr/local/apache-storm-0.9.4/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/apache-storm-0.9.4/lib/ring-servlet-0.3.11.jar:/usr/local/apache-storm-0.9.4/lib/servlet-api-2.5.jar:/usr/local/apache-storm-0.9.4/lib/slf4j-api-1.7.5.jar:/usr/local/apache-storm-0.9.4/lib/snakeyaml-1.11.jar:/usr/local/apache-storm-0.9.4/lib/storm-core-0.9.4.jar:/usr/local/apache-storm-0.9.4/lib/tools.cli-0.2.4.jar:/usr/local/apache-storm-0.9.4/lib/tools.logging-0.2.3.jar:/usr/local/apache-storm-0.9.4/lib/tools.macro-0.1.0.jar:/Users/soushi/Desktop/teststorm/target/teststorm-1.0.jar:/Users/soushi/.storm:/usr/local/apache-storm-0.9.4/bin -Dstorm.jar=/Users/soushi/Desktop/teststorm/target/teststorm-1.0.jar jp.soushi.teststorm.MyTopology
  491  [main] INFO  backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar...
  499  [main] INFO  backtype.storm.StormSubmitter - Uploading topology jar /Users/soushi/Desktop/teststorm/target/teststorm-1.0.jar to assigned location: storm-local/nimbus/inbox/stormjar-b6added0-ffb8-4602-9d89-b567ed87d335.jar
  509  [main] INFO  backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: storm-local/nimbus/inbox/stormjar-b6added0-ffb8-4602-9d89-b567ed87d335.jar
  509  [main] INFO  backtype.storm.StormSubmitter - Submitting topology MyTopology in distributed mode with conf {"topology.workers":31,"topology.debug":true,"topology.max.spout.pending":5000}
  766  [main] INFO  backtype.storm.StormSubmitter - Finished submitting topology: MyTopology
  </code></pre>
  
  <ol start="3">
  <li>I checked the topology using Storm Web UI. After a while, I got NoClassDefFoundError.</li>
  <li>I edited nimbus.yaml, and add <code>java.library.path: "/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib/java"</code>. and make sure /usr/lib/java can be accessible in the docker container.</li>
  </ol>
  
  <p>Here is the error log(worker-6727.log).</p>
  
  <pre><code>2015-04-28T10:05:18.497+0000 b.s.d.executor [ERROR]
  java.lang.NoClassDefFoundError: com/google/gson/Gson
  at jp.soushi.adcontex.SensorSpout.nextTuple(SensorSpout.java:43) ~[stormjar.jar:na]
  at backtype.storm.daemon.executor$fn__4654$fn__4669$fn__4698.invoke(executor.clj:565) ~[storm-core-0.9.4.jar:0.9.4]
  at backtype.storm.util$async_loop$fn__458.invoke(util.clj:463) ~[storm-core-0.9.4.jar:0.9.4]
  at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
  at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
  Caused by: java.lang.ClassNotFoundException: com.google.gson.Gson
  at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_79]
  at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_79]
  at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_79]
  at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_79]
  at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_79]
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) ~[na:1.7.0_79]
  at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_79]
  ... 5 common frames omitted
  2015-04-28T10:05:18.564+0000 b.s.util [ERROR] Halting process: ("Worker died")
  java.lang.RuntimeException: ("Worker died")
  at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:325) [storm-core-0.9.4.jar:0.9.4]
  at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.5.1.jar:na]
  at backtype.storm.daemon.worker$fn__5102$fn__5103.invoke(worker.clj:495) [storm-core-0.9.4.jar:0.9.4]
  at backtype.storm.daemon.executor$mk_executor_data$fn__4555$fn__4556.invoke(executor.clj:240) [storm-core-0.9.4.jar:0.9.4]
  at backtype.storm.util$async_loop$fn__458.invoke(util.clj:473) [storm-core-0.9.4.jar:0.9.4]
  at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
  at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
  </code></pre>
  
  <ul>
  <li>pom.xml
  I tried changing gson's scope(runtime, provided, compile), but it still doesn't work</li>
  </ul>
  
  <hr>
  
  <pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
  &lt;project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
      &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
      &lt;groupId&gt;jp.soushi&lt;/groupId&gt;
      &lt;artifactId&gt;teststorm&lt;/artifactId&gt;
      &lt;version&gt;1.0&lt;/version&gt;
      &lt;packaging&gt;jar&lt;/packaging&gt;
  
      &lt;name&gt;teststorm&lt;/name&gt;
      &lt;url&gt;http://maven.apache.org&lt;/url&gt;
  
      &lt;properties&gt;
          &lt;project.build.sourceEncoding&gt;UTF-8&lt;/project.build.sourceEncoding&gt;
          &lt;maven.compiler.source&gt;1.7&lt;/maven.compiler.source&gt;
          &lt;maven.compiler.target&gt;1.7&lt;/maven.compiler.target&gt;
      &lt;/properties&gt;
  
      &lt;build&gt;
          &lt;plugins&gt;
              &lt;plugin&gt;
                  &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
                  &lt;artifactId&gt;maven-compiler-plugin&lt;/artifactId&gt;
                  &lt;version&gt;3.3&lt;/version&gt;
                  &lt;configuration&gt;
                      &lt;source&gt;1.7&lt;/source&gt;
                      &lt;target&gt;1.7&lt;/target&gt;
                  &lt;/configuration&gt;
              &lt;/plugin&gt;
              &lt;plugin&gt;
                  &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
                  &lt;artifactId&gt;maven-jar-plugin&lt;/artifactId&gt;
                  &lt;configuration&gt;
                      &lt;archive&gt;
                          &lt;manifest&gt;
                              &lt;addClasspath&gt;true&lt;/addClasspath&gt;
                              &lt;classpathPrefix&gt;lib/&lt;/classpathPrefix&gt;
                              &lt;mainClass&gt;jp.soushi.adcontex.AdContexTopology&lt;/mainClass&gt;
                          &lt;/manifest&gt;
                      &lt;/archive&gt;
                  &lt;/configuration&gt;
              &lt;/plugin&gt;
              &lt;plugin&gt;
                  &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
                  &lt;artifactId&gt;maven-dependency-plugin&lt;/artifactId&gt;
                  &lt;executions&gt;
                      &lt;execution&gt;
                          &lt;id&gt;copy&lt;/id&gt;
                          &lt;phase&gt;install&lt;/phase&gt;
                          &lt;goals&gt;
                              &lt;goal&gt;copy-dependencies&lt;/goal&gt;
                          &lt;/goals&gt;
                          &lt;configuration&gt;
                              &lt;outputDirectory&gt;${project.build.directory}/lib&lt;/outputDirectory&gt;
                          &lt;/configuration&gt;
                      &lt;/execution&gt;
                  &lt;/executions&gt;
              &lt;/plugin&gt;
          &lt;/plugins&gt;
      &lt;/build&gt;
  
      &lt;repositories&gt;
          &lt;repository&gt;
              &lt;id&gt;central&lt;/id&gt;
              &lt;url&gt;https://repo1.maven.org/maven2&lt;/url&gt;
              &lt;releases&gt;
                  &lt;enabled&gt;true&lt;/enabled&gt;
              &lt;/releases&gt;
          &lt;/repository&gt;
          &lt;repository&gt;
              &lt;id&gt;github-releases&lt;/id&gt;
              &lt;url&gt;http://oss.sonatype.org/content/repositories/github-releases/&lt;/url&gt;
          &lt;/repository&gt;
          &lt;repository&gt;
              &lt;id&gt;clojars.org&lt;/id&gt;
              &lt;url&gt;http://clojars.org/repo&lt;/url&gt;
          &lt;/repository&gt;
          &lt;repository&gt;
              &lt;id&gt;local-project-libraries&lt;/id&gt;
              &lt;name&gt;Local project libraries&lt;/name&gt;
              &lt;url&gt;file://${project.basedir}/lib&lt;/url&gt;
              &lt;layout&gt;default&lt;/layout&gt;
          &lt;/repository&gt;
      &lt;/repositories&gt;
  
      &lt;dependencies&gt;
          &lt;dependency&gt;
              &lt;groupId&gt;junit&lt;/groupId&gt;
              &lt;artifactId&gt;junit&lt;/artifactId&gt;
              &lt;version&gt;3.8.1&lt;/version&gt;
              &lt;scope&gt;test&lt;/scope&gt;
          &lt;/dependency&gt;
          &lt;dependency&gt;
              &lt;groupId&gt;storm&lt;/groupId&gt;
              &lt;artifactId&gt;storm-lib&lt;/artifactId&gt;
              &lt;version&gt;0.9.0&lt;/version&gt;
              &lt;scope&gt;provided&lt;/scope&gt;
          &lt;/dependency&gt;
          &lt;dependency&gt;
              &lt;groupId&gt;com.google.code.gson&lt;/groupId&gt;
              &lt;artifactId&gt;gson&lt;/artifactId&gt;
              &lt;version&gt;2.3.1&lt;/version&gt;
              &lt;type&gt;jar&lt;/type&gt;
              &lt;scope&gt;runtime&lt;/scope&gt;
          &lt;/dependency&gt;
      &lt;/dependencies&gt;
  &lt;/project&gt;
  </code></pre>
  
  <ul>
  <li>MyTopology.java</li>
  </ul>
  
  <hr>
  
  <pre><code>package jp.soushi.teststorm;
  import backtype.storm.Config;
  import backtype.storm.StormSubmitter;
  import backtype.storm.topology.TopologyBuilder;
  
  public class MyTopology {
    public static void main(String[] args) throws Exception {
      TopologyBuilder builder = new TopologyBuilder();
  
      builder.setSpout("Logs", new LoggerSpout(), 5);
  
      Config conf = new Config();
      conf.setDebug(true);
      conf.setNumWorkers(31);
      conf.setMaxSpoutPending(5000);
      StormSubmitter.submitTopology( "MyTopology", conf, builder.createTopology() );
    }
  }
  </code></pre>
  
  <ul>
  <li>LoggerSpout.java</li>
  </ul>
  
  <hr>
  
  <pre><code>package jp.soushi.teststorm;
  
  import backtype.storm.Config;
  import backtype.storm.spout.SpoutOutputCollector;
  import backtype.storm.task.TopologyContext;
  import backtype.storm.topology.OutputFieldsDeclarer;
  import backtype.storm.topology.base.BaseRichSpout;
  import backtype.storm.tuple.Fields;
  import backtype.storm.tuple.Values;
  import backtype.storm.utils.Utils;
  import com.google.gson.Gson;
  import java.util.HashMap;
  
  import java.util.Map;
  import org.slf4j.Logger;
  import org.slf4j.LoggerFactory;
  
  
  public class LoggerSpout extends BaseRichSpout {
      public static Logger LOG = LoggerFactory.getLogger(LoggerSpout.class);
      boolean _isDistributed;
      SpoutOutputCollector _collector;
  
      public LoggerSpout() {
          this(true);
      }
  
      public LoggerSpout(boolean isDistributed) {
          _isDistributed = isDistributed;
      }
  
      public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) {
          _collector = collector;
      }
  
      public void close() {
  
      }
  
      public void nextTuple() {
          Utils.sleep(100);
          Gson gson = new Gson();
  
          _collector.emit(new Values("Test"));
      }
  
      public void ack(Object msgId) {
  
      }
  
      public void fail(Object msgId) {
  
      }
  
      public void declareOutputFields(OutputFieldsDeclarer declarer) {
          declarer.declare(new Fields("Logs"));
      }
  
      @Override
      public Map&lt;String, Object&gt; getComponentConfiguration() {
          if(!_isDistributed) {
              Map&lt;String, Object&gt; ret = new HashMap&lt;String, Object&gt;();
              ret.put(Config.TOPOLOGY_MAX_TASK_PARALLELISM, 1);
              return ret;
          } else {
              return null;
          }
      }    
  }
  </code></pre>
  
  <p>I found this answer. But I don't understand well.
  <a href="https://stackoverflow.com/questions/10760667/java-lang-noclassdeffounderror-com-google-gson-gson">java.lang.NoClassDefFoundError: com/google/gson/Gson</a></p>
  
  <p>I also read these answers.</p>
  
  <ul>
  <li><a href="https://stackoverflow.com/questions/25590303/getting-java-lang-classnotfoundexception-com-google-gson-gson-for-maven-project">Getting java.lang.ClassNotFoundException: com.google.gson.Gson for maven project</a></li>
  <li><a href="https://stackoverflow.com/questions/11402633/gson-is-not-being-imported-into-the-maven-pproject">GSON is not being imported into the maven pproject</a></li>
  <li><a href="http://www.ciiycode.com/0JSJ6PgUXWxq/javalangnoclassdeffounderror-comgooglegsongsonbuilder-error" rel="nofollow noreferrer">http://www.ciiycode.com/0JSJ6PgUXWxq/javalangnoclassdeffounderror-comgooglegsongsonbuilder-error</a></li>
  </ul>
  
  <p>I spend 3 days to solve that... I have no idea.
  Can anyone provide any solution to this.Thanks in advance.</p>