Update:
Exporting PYTORCH_BUILD_VERSION=1.10.0+cpu and 1.10.0+cu113 works without issues in my current linux container now.
Regarding the scripts mentioned by @ptrblck , I think these scripts are run when promoting built binaries, not when building them. The vision script is under conda, and there isn’t one for torch. Thanks for the help though.