Q:

Spark Documentation : simple example to add list elements

I'm learning Spark and came across this section of the documentation dealing with parallelized collections.  I replicate the following in python from the documentation to perform a Reduce step:
from pyspark import SparkConf, SparkContext

configuration = SparkConf().setMaster("local[4]").setAppName("Test") 
sc = SparkContext(conf = configuration)

data = [1, 2, 3, 4, 5] 
datapar = sc.parallelize(data)

result = datapar.reduce(lambda a, b: a + b) 
print result

The answer I get is 15, which as explained in the document is the addition of all the elements.  However if I modify result to be datapar.reduce(lambda a, b: a) and print result I get 1, and if I modify it to datapar.reduce(lambda a, b: b) I get 5.
How do the values a and b individually return 1 and 5 but added together in my original code result in the addition of all the elements in datapar?  Is their more to the syntax?  Thanks!

A:

This is purely a question on how reduce works.
The left argument of the lambda function (a in this case) represents the aggregated values of applying the lambda function a + b.  The value on the right is the updated value the iterable.  For example:
iterable   a   b   a + b
========  === === =======
   1       -   -     -
   2       1   2     3
   3       3   3     6
   4       6   4    10
   5      10   5    15 <== Final result

This is how it will look with lambda a + b: a
iterable   a   b     a
========  === === =======
   1       -   -     -
   2       1   2     1
   3       1   3     1
   4       1   4     1
   5       1   5     1 <== Final result

...and with lambda a + b: b
iterable   a   b     b
========  === === =======
   1       -   -     -
   2       1   2     2
   3       2   3     3
   4       3   4     4
   5       4   5     5 <== Final result

