I am now 100% certain that it is a problem with cuda 11, When using another GPU I have with cuda10.1 it works fine. I cannot prove it unfortunately, but I now understand that it started when I installed cuda11 and conda install pytorch=10.2 as previous posts here suggested it is fine. I assume it is considered a bug, how can I nail the problem and report it?