Spark, Distributed Systems, Machine Learning, Hadoop, Scalability.
EMR Cluster Performance for various cluster sizes Alternatively, you can execute the PySpark code in the cluster by using spark-submit command, after connecting to master node via SSH. Set DOCKER_IMAGE_NAME & DOCKER_CLIENT_CONFIG env variable Execute .py file with spark-submit & deploy-mode as