@ptrblck ran everything successfully for partial training on the default pytorch docker image (ubuntu 18.04). Going to run the full training tonight and hopefully it continues to work. Still unsure of what caused the issue, but a clean environment seems to help.
Thank you for all of your help!