I was thinking of a cpp stack trace with  gdb  or similar tools. But I understand that it can be tricky on cloud platforms

Oh cool did not realize that was possible. This should be something I can do despite being on cloud since I can reproduce the error on a VM. Honestly I’m really not familiar with Python’s internals or C in general. I found this link: https://wiki.python.org/moin/DebuggingWithGdb 1 that makes it look like I can just install gdb and run my training with that, and when it fails I’ll be able to see the C stack trace. Is that what you’re referring to?
In the mean time I’ll work on getting an MVE (minimum viable error  ) and post it here when I do.