Spark, Distributed Systems, Machine Learning, Hadoop, Scalability.
create a list of lists & flatMap merges the list into one. Task 3: Data Preprocessing Most data preprocessing can be done with DataFrame API using Spark SQL. Spark SQL query executed on Spark Data Frame will be converted into Map and Reduce ops before execution in Spark. Intro to Spark MLLib & ML