I = torch.randn(40,1,60000)

layer1 = nn.Sequential(
            nn.Conv1d(1, 64, 3, 1),
            nn.BatchNorm1d(64),
            nn.ReLU(),
            nn.MaxPool1d(kernel_size=3, stride=2))

layer2 = nn.Sequential(
            nn.Conv1d(64, 100, 5, 1),
            nn.BatchNorm1d(100),
            nn.ReLU(),
            nn.MaxPool1d(kernel_size=3, stride=2))

layer3 = nn.Sequential(
            nn.Conv1d(100, 100, 7, 1),
            nn.BatchNorm1d(100),
            nn.ReLU(),
            nn.MaxPool1d(kernel_size=3, stride=2))

op = layer1(I)
print(op.shape)

op = layer2(op)
print(op.shape)

op = layer3(op)
print(op.shape)

op is now [40, 100, 7494]
rnn = nn.LSTM(7494, 60, 2, bidirectional=True, dropout=0.5, batch_first=True)
op, not_needed = rnn(op)

In the above op is your output. Ignore the variable not_needed for now. op will be of shape (batch, seq, num_directions * hidden_size), which upon checking, it is.
op shape now = [40, 100, 120]
Here’s where I’m not too sure. I have limited experience using LSTMs and looking at my codes, I’ve only used the final sequence. Depending on what you are going to do, you can leave op as is or if you are taking the final sequence only, reshape it as, op = op[:, -1, :].reshape(40, 1, 120)
encoder_layer = nn.TransformerEncoderLayer(d_model=120, dim_feedforward=512, nhead=your_nhead)
op = encoder_layer(op)

op now will have 2 shapes, one if you used all the sequences another if you used only the last.
[40, 100, 120]
[40, 1, 120]
From here, your Linear Layers are [100x120] or [1x120].
The 100 that you mentioned is used as the sequence length for your LSTM.