For anyone still having this error. I ran into the same problem today. The reason was because I was using multiple GPUS,I “fixed” it by limiting the amount of GPU’s to 1. So this probably has something to do with data being spread out on different GPU’s