Please report back if you find that distributed is faster than data parallel (i.e. time it and lets us know). My intuition says that the claim is false but I haven’t seen anywhere on the docs a proper MWE on how to properly use distributed for each individual case in combination with launch.py.