Hi @leoribeiro I think this general solution helps your case.
x = torch.tensor(
       [[  -2.0813,   -1.3431, 0],
        [   1.7704, 0, 0],
        [   0.1229,    1.2149,    0.5455]])
y= torch.tensor(
        [[   1.6999,   -0.7491,   -1.9227, 0],
        [   0.4784,   -0.2545, 0, 0],
        [  -1.5127,   -0.1439,   -0.1829,   -0.3703]])

## first we concat the tensor
concated_tensor = torch.cat([x,y], dim=-1)
## The key is to create this index:
index = torch.tensor([[2,3,4,5], [1,2,3,4], [3,4,5,6]])
## Finally we call
concated_tensor.scatter_(1, index, y)

We finally obtain something like this:
tensor([[  -2.0813,   -1.3431,    1.6999,   -0.7491,   -1.9227, 0, 0],
        [   1.7704,    0.4784,   -0.2545, 0,0,0,0],
        [   0.1229,    1.2149,    0.5455,   -1.5127,   -0.1439,   -0.1829, -0.3703]])

How to create that index tensor?
tmp_idx = torch.bincount((x!=0).nonzero()[:,0], minlength=3) ##3 is batch size
## tmp_idx = tensor([2, 1, 3])
## Create a ones tensor
pad_ones = torch.ones(y.size(0), y.size(1)-1)
##>>> pad_ones
##tensor([[1., 1., 1.],
  ##      [1., 1., 1.],
  ##      [1., 1., 1.]])
update_index = torch.cat([num.unsqueeze(-1), pad_ones.long()], dim=-1)
index = torch.cumsum(update_index,dim=1)

You will end up with this index:
tensor([[2, 3, 4, 5],
        [1, 2, 3, 4],
        [3, 4, 5, 6]])