I’m not sure I fully understood your question, but the following code gives the same output as your loop.
The permute function can be helpful here.
a = torch.rand((4, 3, 50, 50, 50))

b = a.permute(2, 3, 4, 0, 1)
c = b[..., :-1, :].contiguous().view(-1, 3, 3)
d = b[..., -1, :].contiguous().view(-1, 3, 1)

inv = torch.stack([mat.inverse() for mat in torch.unbind(c, 0)])
out = torch.bmm(inv, d)
out = out.transpose(0,1).contiguous().view(3,50,50,50)

Note I still have to unbind once and to loop to compute the inverse.