Spark, Distributed Systems, Machine Learning, Hadoop, Scalability.
lineLengths is not immediately computed, due to laziness. Only when ‘reduce’ is run, Spark breaks the computation into tasks to run on separate machines, to compute total length. lines = sc.textFile("data.txt") lineLengths = lines.map(lambda s: len(s)) totalLength = lineLengths.reduce(lambda a, b: