I ran my script with CUDA_LAUNCH_BLOCKING=1 like this :
CUDA_LAUNCH_BLOCKING=1 python3 imagenet_train.py args
and the error log is as follows :
/home/ShishoSama/anaconda3/lib/python3.6/site-packages/torchvision/transforms/transforms.py:397: UserWarning: The use of the transforms.RandomSizedCrop transform is deprecated, please use transforms.RandomResizedCrop instead.
  "please use transforms.RandomResizedCrop instead.")
/home/ShishoSama/anaconda3/lib/python3.6/site-packages/torchvision/transforms/transforms.py:156: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  "please use transforms.Resize instead.")
 [resnet18] ::   0/100 ----- [[2018-05-03 16:14:05]] [Need: 00:00:00]
i:  0
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.016279220581055
Epoch: [0][0/10010]	Time 8.858 (8.858)	Data 7.036 (7.036)	Loss 7.0163 (7.0163)	Prec@1 0.000 (0.000)	Prec@5 0.000 (0.000)
i:  1
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.077082633972168
i:  2
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.124829292297363
i:  3
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.023961544036865
i:  4
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.081689834594727
i:  5
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.205571174621582
i:  6
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.2464823722839355
i:  7
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.15701150894165
i:  8
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
loss:  torch.Size([1])
input.size():  128
loss.data[0]:  7.310776233673096
i:  9
output:  torch.Size([128, 1000])
output(target_var):  torch.Size([128])
/pytorch/torch/lib/THCUNN/ClassNLLCriterion.cu:101: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [10,0,0] Assertion `t >= 0 && t < n_classes` failed.
THCudaCheck FAIL file=/pytorch/torch/lib/THCUNN/generic/ClassNLLCriterion.cu line=113 error=59 : device-side assert triggered
Traceback (most recent call last):
  File "imagenet_train.py", line 313, in <module>
    main()
  File "imagenet_train.py", line 140, in main
    train(train_loader, model, criterion, optimizer, epoch, log)
  File "imagenet_train.py", line 184, in train
    loss = criterion(output, target_var)
  File "/home/ShishoSama/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ShishoSama/anaconda3/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 679, in forward
    self.ignore_index, self.reduce)
  File "/home/ShishoSama/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1161, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, size_average, ignore_index, reduce)
  File "/home/ShishoSama/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1052, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce)
RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THCUNN/generic/ClassNLLCriterion.cu:113
terminate called after throwing an instance of 'std::runtime_error'
  what():  cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCStorage.c:184
Aborted (core dumped)