Solved by tom in post #2 
              
              
                So you want the tensor x1 to be expanded to 64 x 8 x 1 x 33325, right? 
This is a case where broadcasting isn’t automatic, but if you do x1 = x1.expand(-1, 8, -1, -1, -1), the torch.cat([x, x1], dim=-1) should work. 
Best regards 
Thomas