The version of Pytorch is 0.3.0 post4
/proc/meminfo show:

MemTotal:       131748088 kB
MemFree:        32858344 kB
MemAvailable:   124043064 kB
Buffers:           70512 kB
Cached:         89183512 kB
SwapCached:        77736 kB
Active:         43803028 kB
Inactive:       49002496 kB
Active(anon):    3116556 kB
Inactive(anon):   764844 kB
Active(file):   40686472 kB
Inactive(file): 48237652 kB
Unevictable:      112548 kB
Mlocked:          112548 kB
SwapTotal:       8387580 kB
SwapFree:        7533240 kB
Dirty:                12 kB
Writeback:             0 kB
AnonPages:       3438624 kB
Mapped:          1279120 kB
Shmem:            317048 kB
Slab:            4115636 kB
SReclaimable:    2598468 kB
SUnreclaim:      1517168 kB
KernelStack:       43664 kB
PageTables:        83284 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    74261624 kB
Committed_AS:   65001116 kB
VmallocTotal:   34359738367 kB
VmallocUsed:     1048844 kB
VmallocChunk:   34291571748 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1517568 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:    129799556 kB
DirectMap2M:     4304896 kB
DirectMap1G:     2097152 kB

What is more, when I change the num_workers in DataLoader from 1 to 0. It seems that it can run now!
The shared memory is indeed low. So is the problem with the shared memory?