<p dir="auto">So I was trying to workout why my parallel code was taking so long.<br>
After-all I only sent the big datastructures once, though a closure as the function that was mapped over.<br>
That should happen, once (I thought), since the function is constant<br>
Not so</p>
<p dir="auto">MWE:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="addprocs(5)

immutable Foo
    bar::Char
end

#HACK: lets debug what is being serialised by overloading the calls
function Base.serialize(s::Base.SerializationState, x::Foo)
    tic()
    Base.Serializer.serialize_any(s,x)
    tt=toq()
    open(&quot;ser_log.txt&quot;,&quot;a&quot;) do fp
        println(fp, tt)
    end
end

function test()
    st = Foo(rand('a':'z')) 
    pmap(r-&gt;string(st.bar)^r, 1:100)
    #Base.pgenerate(default_worker_pool(), r-&gt;string(st.bar)^r, 1:100) |&gt; collect
end"><pre class="notranslate"><code class="notranslate">addprocs(5)

immutable Foo
    bar::Char
end

#HACK: lets debug what is being serialised by overloading the calls
function Base.serialize(s::Base.SerializationState, x::Foo)
    tic()
    Base.Serializer.serialize_any(s,x)
    tt=toq()
    open("ser_log.txt","a") do fp
        println(fp, tt)
    end
end

function test()
    st = Foo(rand('a':'z')) 
    pmap(r-&gt;string(st.bar)^r, 1:100)
    #Base.pgenerate(default_worker_pool(), r-&gt;string(st.bar)^r, 1:100) |&gt; collect
end
</code></pre></div>
<p dir="auto">Then running the function and counting the lines in the log:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="test()
run(`wc -l ser_log.txt`)
OUT&gt; 17"><pre class="notranslate"><code class="notranslate">test()
run(`wc -l ser_log.txt`)
OUT&gt; 17
</code></pre></div>
<p dir="auto">So it was serialized 17 times for pmap.<br>
If is switch to <code class="notranslate">pgenerate</code> is it 18 times (so about the same).<br>
I believe that after the batchsplit step is done that is once serialisation of the closure, per batch that was sent.<br>
It only need to be  serialized once.</p>
<p dir="auto">(in my nonMWE, it is happening millions of times, and takes 6 seconds a piece...)</p>
<hr>
<p dir="auto">I suspect this is already known, but I can't find an issue for it, so maybe not.</p>
<p dir="auto">see also:</p>
<ul dir="auto">
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="91912504" data-permission-text="Title is private" data-url="https://github.com/JuliaLang/julia/issues/11938" data-hovercard-type="issue" data-hovercard-url="/JuliaLang/julia/issues/11938/hovercard" href="https://github.com/JuliaLang/julia/issues/11938">#11938</a></li>
<li><a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="154380639" data-permission-text="Title is private" data-url="https://github.com/JuliaLang/julia/issues/16322" data-hovercard-type="issue" data-hovercard-url="/JuliaLang/julia/issues/16322/hovercard" href="https://github.com/JuliaLang/julia/issues/16322">#16322</a></li>
</ul>
<hr>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="versioninfo()
Julia Version 0.5.0-dev+3928
Commit fdc4a85 (2016-05-06 04:46 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: AMD Opteron 63xx class CPU
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Piledriver)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, bdver2"><pre class="notranslate"><code class="notranslate">versioninfo()
Julia Version 0.5.0-dev+3928
Commit fdc4a85 (2016-05-06 04:46 UTC)
Platform Info:
  System: Linux (x86_64-linux-gnu)
  CPU: AMD Opteron 63xx class CPU
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Piledriver)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, bdver2
</code></pre></div>
<p dir="auto">[Originally posted on -dev by Fermat 618: https://groups.google.com/d/topic/julia-dev/MOt74gNrc9E/discussion]</p>
<p dir="auto">When I assign a lambda to a function,</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="julia&gt; f(x) = x + 2

julia&gt; f = (x) -&gt; x + 20
Warning: redefinition of constant f ignored.
#&lt;function&gt;

julia&gt; f(3)
5"><pre class="notranslate"><code class="notranslate">julia&gt; f(x) = x + 2

julia&gt; f = (x) -&gt; x + 20
Warning: redefinition of constant f ignored.
#&lt;function&gt;

julia&gt; f(3)
5
</code></pre></div>
<p dir="auto">I find  neither an Error is raised nor the assignment take effect.</p>
<p dir="auto">It is very dangerous to ignore an error. A program that returns a wrong<br>
answer is much more worse than a program that fails.</p>
<p dir="auto">When I write something, I expected it to take effect. If it do not, I<br>
expect an error. A warning is reasonable when someone write</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="     if (a = b) { xxx; }"><pre class="notranslate"><code class="notranslate">     if (a = b) { xxx; }
</code></pre></div>
<p dir="auto">in C. Even though <code class="notranslate">if (a == b)</code> is likely what one want, one can still<br>
ignore the warning when he/she is sure <code class="notranslate">if (a = b)</code> is wanted. But this<br>
is definitely not the case.</p>
<p dir="auto">In my opinion, a well designed programming language should give as less<br>
[warnings] as possible.</p>