Here is some more info on my problem.  I am trying to get the output of an LSTM to pass into a Linear layer.  I can’t get it to work for the life of me, and that is why I am trying to make sure I have my sizes straight in my first post above.
Here is the model:
class LSTM(nn.Module):

    def __init__(self, num_classes, input_size, hidden_size, num_layers, batch_size):
        super(LSTM, self).__init__()
        
        self.num_classes = num_classes
        self.input_size = input_size
        self.hidden_size = hidden_size
        self.num_layers = num_layers
        self.batch_size = batch_size
        
        self.dropout = nn.Dropout(p=0.2)
        
        self.lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size,
                            num_layers=num_layers, batch_first=True, dropout = 0.25)

        self.fc = nn.Linear(hidden_size, num_classes)

    def init_hidden(self):
        self.h_0 = torch.zeros(self.num_layers, self.batch_size, self.hidden_size).to(device)
        self.c_0 = torch.zeros(self.num_layers, self.batch_size, self.hidden_size).to(device)     
        
    def forward(self, x):                                             # x torch.Size([20, 28, 30495])
        lstm_output, (h_n, c_n) = self.lstm(x, (self.h_0, self.c_0))  
                                                                               # lstm_output torch.Size([20, 28, 512])
                                                                              # h_n torch.Size([1, 20, 512])
    
        lstm_output = torch.flatten(lstm_output, start_dim=1)
        out = self.fc(lstm_output)
        out = self.dropout(out)
        return out

Originally I got an error that lstm_output was the wrong size
RuntimeError: The size of tensor a (28) must match the size of tensor b (20) at non-singleton dimension 1


So that is why I used flatten.  But with the code the way it is above, it gives another error:
RuntimeError: size mismatch, m1: [20 x 14336], m2: [512 x 30490] at /tmp/pip-req-build-8whce6xx/aten/src/THC/generic/THCTensorMathBlas.cu:283

If I send h_n into the linear layer instead of lstm_output, it works fine, in fact I can just use view to flatten h_n and that works.  But I don’t know that I want to send h_n.  It would seem the proper object to send is lstm_output.
Obviously I have a mismatch or something somewhere.  If I set batch_size to 28, everything works with sending lstm_output into the linear layer and I don’t even have to use flatten.  But thats why I think I have something mixed up because I should be able to set batch_size to whatever I want correct?