Spark, Scala, Java, Big Data, Data Science.
Job submission would be managed by an orchestration tool like Apache Airflow. Unit tests are essential and a framework such as scalatest can be employed for writing and executing tests. Spark Project setup with IntelliJ IDEA IDE Using an IDE such as IntelliJ IDEA with Scala and sbt plugins, has the