Hey @bing, thanks for the question, and sorry about the delay. Could you please add the “distributed” tag to DP/DDP/RPC/C10D related questions in the future? The PT Distributed team is actively monitoring that tag, but we might miss uncategorized questions.