<p dir="auto">I have run into a strange problem using DistributedDataParallel. I have solved it - but this looks like a bug to me.</p>
<p dir="auto">The error I am getting is:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="... in spawnv_passfds
    False, False, None)
ValueError: bad value(s) in fds_to_keep"><pre class="notranslate"><code class="notranslate">... in spawnv_passfds
    False, False, None)
ValueError: bad value(s) in fds_to_keep
</code></pre></div>
<p dir="auto">This occurs during the spawn() call from pytorch-lightning. The model I am using works in all other modes, and an identical model, trainer setup works in ddp, too.</p>
<p dir="auto">After some hair pulling, I found the problem. During the construction of the new model, and before creating trainer, etc, I store the .data property of a Parameter:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="self.class_p        = nn.Parameter(torch.Tensor(np.ones(self.data.num_classes) * np.log(1.0)), requires_grad=True)
self.class_p_t      = self.class_p.data # this is the offending line"><pre class="notranslate"><code class="notranslate">self.class_p        = nn.Parameter(torch.Tensor(np.ones(self.data.num_classes) * np.log(1.0)), requires_grad=True)
self.class_p_t      = self.class_p.data # this is the offending line
</code></pre></div>
<p dir="auto">I do NOT actually do anything with self.class_p_t - I assigned it in anticipation of logging these values to monitor progress of self learning curriculum learning class weights.</p>
<p dir="auto">If the assignment above is present, I get the above failure. Without it, things work fine.</p>
<p dir="auto">If this is something you expect, I'd appreciate knowing why - if it's a bug, thought you'd like to know.</p>
<p dir="auto">For the record:Ubuntu 19.10, python 3.7.5, pytorch 1.4, venv, pytorch-lightning 0.7.1<br>
Note that this is a fully up to date ubuntu ...</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/pietern/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/pietern">@pietern</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mrshenli/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mrshenli">@mrshenli</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/pritamdamania87/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/pritamdamania87">@pritamdamania87</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zhaojuanmao/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zhaojuanmao">@zhaojuanmao</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/satgera/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/satgera">@satgera</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/rohan-varma/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/rohan-varma">@rohan-varma</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gqchen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gqchen">@gqchen</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/aazzolini/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/aazzolini">@aazzolini</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/xush6528/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/xush6528">@xush6528</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/osalpekar/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/osalpekar">@osalpekar</a></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import torch

fig = plt.figure()
plt.plot([1,2])
fig.savefig('test.pdf')"><pre class="notranslate"><code class="notranslate">import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
import torch

fig = plt.figure()
plt.plot([1,2])
fig.savefig('test.pdf')
</code></pre></div>
<p dir="auto">Running this simple python code will have error <code class="notranslate">*** Error in 'python': free(): invalid pointer: 0x00007f5ba8129ac0 ***</code> (omit the backtrace output). It breaks at <code class="notranslate">fig.savefig('test.pdf')</code>.</p>
<p dir="auto">OS Ubuntu 16.04.2 LTS, Python version 2.7.12, PyTorch version 0.1.12_2, Matplotlib version 2.0.2</p>