Thank you!
On torch 1.6, the following works:
#%env NCCL_DEBUG=INFO
import torch
a,b,c,d = 300,400,500,6000
x = torch.rand(a, b, c).cuda()

model = torch.nn.Sequential(torch.nn.Linear(c, d))
model = torch.nn.DataParallel(model.cuda(), device_ids=[0,1,2,3])

%timeit -n 5 y = model(x)

But on torch 1.8, Sagemaker prints the following warning and
INFO profiler_config_parser.py:102] Unable to find config at /opt/ml/input/config/profilerconfig.json. Profiler is disabled.
and this error
RuntimeError: NCCL Error 2: unhandled system error
Do you have any idea what causes the trouble?