Update: While this code was running neatless in version ‘1.0.1’, it somehow fails under ‘1.0.1.post2’. The following error appears after different time periods:
KeyError: 'Traceback (most recent call last):\n File "/home/catrueeb/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in _worker_loop\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/catrueeb/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 138, in <listcomp>\n samples = collate_fn([dataset[i] for i in batch_indices])\n File "/home/catrueeb/utils/cnn_utils.py", line 45, in __getitem__\n input = self.transform(torch.tensor(self.dataset[\'inputs\'][index, :, :].astype(\'float32\')))\n File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper\n File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper\n File "/home/catrueeb/.local/lib/python3.7/site-packages/h5py/_hl/group.py", line 262, in __getitem__\n oid = h5o.open(self.id, self._e(name), lapl=self._lapl)\n File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper\n File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper\n File "h5py/h5o.pyx", line 190, in h5py.h5o.open\nKeyError: \'Unable to open object (wrong B-tree signature)\'\n'
Edit:
torch.multiprocessing.set_start_method('spawn')
solved it