Yes, i split tensor [4,5, 120] and pass each sentence to encoder.
Here’s final version of correct encoder:
class CNNEncoder(nn.Module):
    def __init__(self, embed_dim: int, vocab_size: int):
        super(CNNEncoder, self).__init__()
        self.embed = nn.Embedding(vocab_size, embed_dim)
        self.conv1 = nn.Conv2d(1, 100, (2, embed_dim))

    def forward(self, x):
        x = self.embed(x)
        x = x.unsqueeze(0)
        x = x.unsqueeze(0)
        x = self.conv1(x)
        x = x.squeeze(3)
        x = torch.tanh(x)
        x = F.avg_pool1d(x, x.size()[2]).squeeze(2)
        return x

I’m not sure if this method of splitting tensor is ok.
Here it is (latest update):
for class_row in support_set.split(1):
    encoded_sents = []
    for sentence in class_row.squeeze_().split(1):
        encoded_sents.append(self.cnn_encoder.forward(sentence.squeeze_()))
     encoded_support_set.append(encoded_sents)

class_tensors = []
for i in range(support_set.shape[0]):
    sentence_tensors = []
    for j in range(support_set.shape[1]):
        sentence_tensors.append(torch.tensor(encoded_support_set[i][j].clone().detach().requires_grad_(True)))
    class_tensors.append(torch.cat(sentence_tensors))

support_set = torch.stack(class_tensors)