Pyspark, Mongodb, AWS, Machine Learning.
has a very convenient support with pandas dataframes. Transition from MongoDB collection to Spark RDD is as easy as: # create train and test rdds train_year_rdd = sc.parallelize(list(train_year_data)) test_year_rdd = sc.parallelize(list(test_year_data)) We will create a 1d feature vector out of