<h2 dir="auto">🐛 Bug</h2>
<p dir="auto">I’d like to report a weird behaviour in 1.0 that I could resolve by only going back to 0.4. I am training a network on quite a big data set (35 GB) and use 4 gpus by applying the command<br>
torch.nn.DataParallel(model).cuda()<br>
Further I am having a big batch size (&gt;1000) which makes the command<br>
torch.multiprocessing.set_sharing_strategy(‘file_system’). I have num_workers=16 in the dataloader<br>
necessary. Now the trouble begins: every epoch my /dev/shm increases by ca. 3GB. At some point it is full and my process crashes. I tried 1.0.0 and 1.0.1 but both showed this bahaviour. Pytorch 0.4 does not have this problem, /dev/shm is never above 1GB.</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<ol dir="auto">
<li>Train network on big data set with data.Dataloader with big batch size, for which you require torch.multiprocessing.set_sharing_strategy('file_system') and Dataparallel</li>
<li>Observe /dev/shm until it is full</li>
</ol>

<h2 dir="auto">Expected behavior</h2>
<p dir="auto">In 0.4 version /dev/shm is not filling up and there is not crash</p>
<ul dir="auto">
<li>PyTorch Version (e.g., 1.0.1 and 1.0.0 vs. 0.4.0):</li>
<li>OS (e.g., Linux): Linux</li>
<li>How you installed PyTorch (<code class="notranslate">conda</code>, <code class="notranslate">pip</code>, source): conda</li>
<li>Build command you used (if compiling from source):</li>
<li>Python version: 2.7</li>
<li>CUDA/cuDNN version: 10</li>
<li>GPU models and configuration:</li>
<li>Any other relevant information:</li>
</ul>
<p dir="auto">Importing sklearn after importing torch cases segmentation fault. I have tried reinstalling both scikit-learn and pytorch. I use pip (not anaconda)</p>
<p dir="auto">A) Works</p>
<p dir="auto">ipython<br>
Python 2.7.12 (default, Nov 19 2016, 06:48:10)<br>
Type "copyright", "credits" or "license" for more information.</p>
<p dir="auto">IPython 5.1.0 -- An enhanced Interactive Python.<br>
?         -&gt; Introduction and overview of IPython's features.<br>
%quickref -&gt; Quick reference.<br>
help      -&gt; Python's own help system.<br>
object?   -&gt; Details about 'object', use 'object??' for extra details.</p>
<p dir="auto">In [1]: import sklearn</p>
<p dir="auto">In [2]: import torch</p>
<p dir="auto">In [3]: print sklearn.<strong>version</strong><br>
0.18.1</p>
<p dir="auto">B) Segmentation Fault</p>
<p dir="auto">ipython<br>
Python 2.7.12 (default, Nov 19 2016, 06:48:10)<br>
Type "copyright", "credits" or "license" for more information.</p>
<p dir="auto">IPython 5.1.0 -- An enhanced Interactive Python.<br>
?         -&gt; Introduction and overview of IPython's features.<br>
%quickref -&gt; Quick reference.<br>
help      -&gt; Python's own help system.<br>
object?   -&gt; Details about 'object', use 'object??' for extra details.</p>
<p dir="auto">In [1]: import torch</p>
<p dir="auto">In [2]: import sklearn<br>
Segmentation fault (core dumped)</p>