Thanks.
So, this means, three computation graphs will be computed in my example, one for each forward pass.
Then, when we do loss.backward, the gradients will be computed for each computation graph using the same loss.
Finally, the optimizer will update the weights using all three graphs.
Then, the question is, would this be equivalent to doing a single forward pass on the concatenated inputs and then doing loss.backward (which is the intended outcome)?
Even if they are mathematically equivalent (? need to check), the three computation graphs make it inefficient.