import torch 
from torch.autograd import Variable
from torch.nn.functional import sigmoid

x = Variable(torch.randn(10,3),requires_grad = False)
w1 = Variable(torch.randn(3,5),requires_grad = True)
w2 = Variable(torch.randn(5,7),requires_grad = True)
z=Variable(torch.randn(1,1),requires_grad = True)
L = (sigmoid(x.mm(w1*z))).mm(w2)

So the key idea here is that if you were to do this in a for loop, you’d be computing L 10 * 7 = 70 times.
Instead of doing this in a for loop, we’d want to batch the computation.
We can define a new Variable, zz, that will hold the derivative matrix. It will eventually be size (10, 7),
but for now we’ll make it a flat size 70 so that we can perform batch multiply operations with it.
Code as follows:
zz = Variable(z.data.expand(70, 1, 1), requires_grad=True)
batched_x = x.expand(70, 10, 3)
batched_w1 = w1.expand(70, 3, 5)
batched_w2 = w2.expand(70, 5, 7)
batched_L = (sigmoid(batched_x.bmm(batched_w1*zz))).bmm(batched_w2)
out = batched_L.view(70, 70).trace()
out.backward()
zz.grad.view(10, 7)