title: How to run Spark on Docker using bitnami docker-compose?
body: <p>I'm trying to run spark with bitnami docker-compose</p>
  <p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
  <div class="snippet-code">
  <pre class="snippet-code-html lang-html prettyprint-override"><code>version: '2'
  
  services:
    spark:
      image: docker.io/bitnami/spark:3
      environment:
        - SPARK_MODE=master
        - SPARK_RPC_AUTHENTICATION_ENABLED=no
        - SPARK_RPC_ENCRYPTION_ENABLED=no
        - SPARK_LOCAL_STORAGE_ENCRYPTION_ENABLED=no
        - SPARK_SSL_ENABLED=no
      ports:
        - '8080:8080'
        - '7077:7077'
    spark-worker-1:
      image: docker.io/bitnami/spark:3
      environment:
        - SPARK_MODE=worker
        - SPARK_MASTER_URL=spark://spark:7077
        - SPARK_WORKER_MEMORY=1G
        - SPARK_WORKER_CORES=1
        - SPARK_RPC_AUTHENTICATION_ENABLED=no
        - SPARK_RPC_ENCRYPTION_ENABLED=no
        - SPARK_LOCAL_STORAGE_ENCRYPTION_ENABLED=no
        - SPARK_SSL_ENABLED=no
    spark-worker-2:
      image: docker.io/bitnami/spark:3
      environment:
        - SPARK_MODE=worker
        - SPARK_MASTER_URL=spark://spark:7077
        - SPARK_WORKER_MEMORY=1G
        - SPARK_WORKER_CORES=1
        - SPARK_RPC_AUTHENTICATION_ENABLED=no
        - SPARK_RPC_ENCRYPTION_ENABLED=no
        - SPARK_LOCAL_STORAGE_ENCRYPTION_ENABLED=no
        - SPARK_SSL_ENABLED=no</code></pre>
  </div>
  </div>
  </p>
  <p>I'm able to run spark job when I attach to the container, but I'm unable to create a spark session from the host itself</p>
  <pre><code>from pyspark.sql import SparkSession
  spark = SparkSession.builder.master(&quot;spark://localhost:7077&quot;).appName(&quot;test&quot;).getOrCreate()
  </code></pre>
  <p>I also tried with the container ip:</p>
  <pre><code>from pyspark.sql import SparkSession
  spark = SparkSession.builder.master(&quot;spark://172.19.0.4:7077&quot;).appName(&quot;test&quot;).getOrCreate()
  </code></pre>
  <p>but anything I try I'm getting the same error:</p>
  <pre><code>JAVA_HOME is not set
  Exception: Java gateway process exited before sending its port number
  </code></pre>
  <p>what am I doing wrong?
  OS is ubuntu 20.04</p>