Spark, Distributed Systems, Machine Learning, Hadoop, Scalability.
Notebook-scoped docker in EMR Notebooks to resolve dependencies. Now you can execute PySpark code in subsequent cells. from pyspark.sql import SparkSession spark = SparkSession.builder.appName("docker-numpy").getOrCreate() sc = spark.sparkContext import numpy as np print(np.__version__) Ideally,