I don’t believe it is possible to do async hog wild training with gpu. The point with that type of training is to exploit some of the benefits CPU use has over gpu. You could do a like-a3c batch training where training is parallelized i.e. but global model is updated synchronously not asynchronously.
Ps there is no a3c Gpu hogwild training in tensorflow or pytorch on github to my knowledge