Interesting, could you try and isolate this to a self-contained minimal training/val loop that still causes the segfault (it can have random training data if that works)? I will see if I can reproduce it on similar hardware.