Solved by albanD in post #10 
              
              
                The problem is not the product but that you only want to gradients corresponding to part of it. In some sense, you have K different loss functions. So if you want the gradients for each of them you need to do K backwards. I don’t think there is any way around this.