Hi,

Yes dropout layers and any randomness in general (like random sampler to load data or model initialization) will give different values at every run. If you seed them, you can still get different results if your use them differently from one version to the other (like more parameters in your model mean that more random numbers will be drawn) or if you use non-deterministic ops (see note in the doc here: https://pytorch.org/docs/stable/notes/randomness.html).
Not sure why this should be true? if you just compute the same loss twice then backward then yes it should be true. But you seem to be doing fancy stuff when the embed_grad is provided so I guess you will get a different loss.
It looks like it would do the same yes as you simply accumulate the gradients in to steps.