Hello everyone,
I have a tensor of size (batch_size, seq_length, embed_dim) and after passing through a linear layer I would like to get an input of  (batch_size, latent_dim). The part of the code is shown below
class TransformerEncoder(nn.Module):
    
    def __init__(self, head, embedding_size, pos_embed, latent_dim = 512):
        super(TransformerEncoder, self).__init__()
        self.head = head
        self.embedding_size = embedding_size
        self.pos_embed = pos_embed 
        self.self_attention = SelfAttention(head, embedding_size)
        self.fc1 = nn.Linear(embedding_size, latent_dim)
        self.fc2 = nn.Linear(embedding_size, latent_dim)
        self.dropout = nn.Dropout(0.1)
        
    def forward(self, x):
        batch_size, max_words = x.size()[:2]
        x = self.self_attention(x) 
        x = self.dropout(x)
        pos_x = self.pos_embed.get_postional_embeddings()
        x = (x + pos_x)
        x = x.view(batch_size, -1)
        mean, logvar = self.fc1(x), self.fc2(x)
        return mean, logvar

the variable x is of 3 dimensions, I don’t know the value of batch_size and seq_length until the forward propagation starts. So there will be a size mismatch in the operation self.fc1(x) and in self.fc2(x). One solution I can think of is to assign a large value to seq_length and pad all the inputs to that length. But this would slow down the learning process. Is there anything else that can be done?
Thanks