I just test it again on two pytorch settings (one being the latest)

pytorch 1.7.1 py38_cuda11.0.221_cudnn8.0.5_0
pytorch 1.7.0 py38_cuda11.0.221_cudnn8.0.3_0
and they also encounter the same error.

One situation is a bit strange to me.
When I set False to the anomaly detection mode, I am able to run with the main.py script in the mnist folder from pytorch example git repo.
However, in the same setting, error occurs if I run the main.py script in the word_language_model folder.
The error is as follow (main.py refers to the one under word_language_model)
Traceback (most recent call last):
File “main.py”, line 217, in 
train()
File “main.py”, line 181, in train
torch.nn.utils.clip_grad_norm_(model.parameters(), args.clip)
File “/home/yangle/anaconda3/envs/pytorch_latest/lib/python3.8/site-packages/torch/nn/utils/clip_grad.py”, line 38, in clip_grad_norm_
if clip_coef < 1:
RuntimeError: CUDA error: an illegal memory access was encountered