So my variant of your problem is: when I compile (64-bit) PyTorch on the RaspPi (with 4GB), most of the time, I can use 4 cores (-j4). So I compile with that until I hit OOM, then I run it with -j1 through the large source files, hit ^C when I get bored and go back to -j4 for the remainder. Seems to work reasonably well.