Solved by tom in post #2 
              
              
                Depending on your style preferences, there are 
x = torch.empty(a.size(0), a.size(1), 2*a.size(2), a.size(3), dtype=a.dtype, device=a.device)
x[:, :, 0::2, :] = a
x[:, :, 1::2, :] = b

and 
x = torch.stack([a, b], 3).view(a.size(0), a.size(1), 2*a.size(2), a.size(3))

you could try. 
The latter work…