Spark, Distributed Systems, Machine Learning, Hadoop, Scalability.
Pipeline Spark MLLib is a sklearn-inspired library which contains distributed implementation of popular ML algorithms. The main difference with sklearn is the usage of sc.parallelize() function to split data across multiple boxes. All the steps required to convert the raw data on disk to a final