Hey @Feywell
By “change https://github.com/ZPdesu/SEAN distributed training”, which distributed training API are you referring to (e.g., DistributedDataParallel, c10d, RPC)?
Could you please share the code that uses distributed APIs?