Hogwild training is not made to run on gpu as with cuda the tensors have locks sets in place in most likely will end up in model parameters shared in corrupted state which I don’t think cuda will allow. You could set up a pool of process that will organize them for you and should run on gpu but thats not hogwild training if your looking for that