Thanks for your quick reply!
Your solution works! The problem in my case is just, that H & W are of different sizes. I think i did not explain my problem properly, I am sorry. My two tensors a and b are the same size:
B, H, W = 4, 128, 416

a = torch.randn(B, 3, 3, H, W)
b = torch.randn(B, 3, 3, H, W)


I now tried a different approach, by transposing the tensors before the multiplication to shift them to the sizes [B, H, W, 3, 3):
b = torch.transpose(b, 1, 3)
b = torch.transpose(b, 2, 4)
b.shape
>>> torch.Size([4, 128, 416, 3, 3])

a = torch.transpose(a, 1, 3)
a = torch.transpose(a, 2, 4)
a.shape
>>> torch.Size([4, 128, 416, 3, 3])

c = torch.matmul(a, torch.transpose(b, 3, 4))
c.shape
>>>torch.Size([4, 128, 416, 3, 3])


… and then transposing c back to [B, 3, 3, H, W]. But I am not sure if this somehow messes with the whole multiplication?