Created on 2016-12-29 21:57 by Jose Miguel Colella, last changed 2020-09-18 16:03 by iritkatriel.
Hello I am trying to use the callback for the map_async method for Pool, but have found a bug. In the below code, only the print statement is carried out, the return is completely ignored. Is this working as designed or is this a bug?
from multiprocessing import Pool
def f(x):
return x * x
def s(x):
print(f'Here: {x}')
return type(x)
if __name__ == '__main__':
with Pool(5) as p:
result = p.map_async(f, [1, 2, 3], callback=s)
q = result.get()
print(q)
The result is:
Here: [1, 4, 9]
[1, 4, 9]
This appears to be working as designed. The purpose of the callback is to receive a single argument (nominally, the ready result from the map operation) and perform its task (whatever it may be) very quickly so as not to further delay the handing back of results from the map operation.
Attempting to read into your comments, it sounds like you were perhaps expecting the return from the callback to appear somewhere? If that was your expectation then no, that is not the purpose of a callback here. Any transformation of input data to the final output data is performed by the function being applied by the map. While there are many potential uses for a callback, its stated requirement is that it be as quick an operation as possible as opposed to something that further transforms the results or anything like a "reduce" operation which is not necessarily quick.
If you think this needs clarification in the documentation, please suggest something?
Hello David,
Thanks for your response. Improvements to the documentation could clear this misunderstanding. I had initially believed that after transforming with the function passed to the map, it would use the callback on each of the result arguments.
Just to understand the use case of the callback. So basically it should not return anything and be a simple print?