My goal is to concatenate my sentence embedding with shape [128, 20, 16] with the output of the averagepool with shape [128, 10, 10] at position 2 which is 10 + 16. The output of this concatenation is given as input to the next Conv1d layer.
torch.cat((embed.view(-1, sent.shape[1], embed.shape[2]), avg_pool_level_1), 2)

where shape of embed is torch.Size([128, 20, 16]) and shape of avg_pool_1 is torch.Size([128, 10, 10])