Yes, you could use those operations… However I don’t know if nn.Linear would be adequate to construct your desired module. You’ll have to define a few nn.Parameters corresponding to the W matrices and b vectors and then use nn.conv2d together with torch.bmm in order to achieve those equations