If these issues are specific to a particular GPU, I would assume it might be a hardware defect.
Are you able to reproduce the illegal memory access in every run on this device and are other GPUs working fine?