Pyspark, Mongodb, AWS, Machine Learning.
spark represents data and stores it in partitions. Data can come from either text, csv, json format or a database. I couldn’t find a direct way to convert H5 files into an RDD but this link https://www.hdfgroup.org/2015/03/from-hdf5-datasets-to-apache-spark-rdds/ inspired me to write my own script