I’m not sure, how you would like to fill dim2 in your first tensor, but if you just want to repeat the values, this code would work:
a = torch.randn(16, 512)
b = torch.randn(16, 32, 2048)
a = a.unsqueeze(2).expand(-1, -1, 2048)
c = torch.cat((a, b), dim=1)
print(c.shape)
> torch.Size([16, 544, 2048])