The problem was that I am using a library for logging my data (Weights & Biases) that I thought worked fully asynchronously and online, but it actually writes the data to a file on disk before logging it to their servers. This created lots of accesses to the hard drive and, in the computers that were not equipped with SSD, slowed the process down quite a bit. Mystery solved.