Okay so I have been racking my brain trying to figure out why sometimes the build successfully passes and why sometimes it doesn’t. This apparently happens when “mkldnn is out of sync”, the following few git commands solves this problem:
git submodule sync
git submodule update --init --recursive

source: https://github.com/pytorch/pytorch/issues/35602 36