Spark, Sql, Pandas, Python.
user_id and apply the get_median function. Prepartition and Spark to get the medians In this example, we get the median measurement of each user. As the data size gets larger, more benefits will be seen from the parallelization. In an example notebook we have, the Pandas engine took around 520