Hey @alet, can you share the implementation of modified model? Especially the “optimizable conditional normalization layers”. Would I be correct if I assume the program is using DDP both before and after the modification?