Spark, Distributed Systems, Machine Learning, Hadoop, Scalability.
in-memory operations. With the help of Mesos — a distributed system kernel — Spark caches the intermediate data set after each iteration. Since output of each iteration is stored in RDD, only 1 disk read and write operation is required to complete all iterations of SGD. Spark is built on Resilient