ChickenTarm:

I needed to set the random seed that samples the probability of which alternating loss it will perform


This sounds correct, there would probably be a hang if a different alternating loss is used on different ranks.