Thanks for your reply @vdw.  The code you shared is very helpful.
The main thing missing from my fully connected layer input size calculation above is that I interpreted “L in” from the nn.Conv1d docs as the input channels size (in my case embedding_dim).  I see your code uses the max sequence length as “L in” here.
Screen Shot 2021-03-04 at 9.13.55 AM859×113 6.01 KB
I agree that my code above could be refactored to remove repeated lines.  Here’s an updated version, organized much like the code you shared.  I also borrowed the idea of concatenating the outputs of each CNN layer which I’ve seen in other code (follow-up question on this below if you have the chance/are interested).
class DeepCNN(nn.Module):
    """
    Borrows from code here:
    https://github.com/chrisvdweth/ml-toolkit/blob/master/pytorch/models/text/classifier/cnn.py
    """

    def __init__(
        self,
        n_classes,
        max_seq_len,
        embedding_matrix,
        kernel_sizes,
        conv_dropout_rates,
        n_fc_neurons_l0,
        n_fc_neurons_l1,
        fc_dropout_rate_l0,
        fc_dropout_rate_l1,
        pad_idx
    ):
        super(DeepCNN, self).__init__()
        
        self.embedding_dim = embedding_matrix.shape[1]
        self.flatten_size = 0
        self.conv_padding = 0
        self.conv_stride = 1
        self.maxpool_kernel_size = 2
        self.maxpool_padding = 0
        self.maxpool_dilation = 1
        
        # Would like to be able to increase these at each layer
        self.in_channels = [
            self.embedding_dim,
            self.embedding_dim,
            self.embedding_dim
        ]
        
        self.out_channels = [
            self.embedding_dim,
            self.embedding_dim,
            self.embedding_dim
        ]
        
        # Embedding layer
        self.embedding = nn.Embedding.from_pretrained(
            embeddings=embedding_matrix,
            freeze=False,
            padding_idx=pad_idx,
            max_norm=None,
            norm_type=2,
            scale_grad_by_freq=False,
            sparse=False
        )
        
        # Iterate through kernels and channels to set layers and compute output sizes
        self.conv_layers = nn.ModuleDict()
        self.dropout_layers = nn.ModuleDict()
        self.maxpool_layers = nn.ModuleDict()
        for i, (k, ic, oc, p) in enumerate(zip(kernel_sizes, self.in_channels, self.out_channels, conv_dropout_rates)):
            
            # Set conv layers
            self.conv_layers[f'conv_{i}'] = nn.Conv1d(
                in_channels=ic,
                out_channels=oc,
                kernel_size=k,
                stride=self.conv_stride,
                padding=self.conv_padding
            )
            
            # Set dropout layers
            self.dropout_layers[f'dropout_{i}'] = nn.Dropout(p=p)
            
            # Set maxpool layers
            self.maxpool_layers[f'maxpool_{i}'] = nn.MaxPool1d(
                kernel_size=self.maxpool_kernel_size,
                stride=self.maxpool_kernel_size,
                padding=self.maxpool_padding
            )
            
            # Calculate conv output size
            conv_out_size = self._calc_conv_output_size(
                seq_len=max_seq_len,
                kernel_size=k,
                stride=self.conv_stride,
                padding=self.conv_padding
            )
            
            # Calculate maxpool output size
            maxpool_out_size = self._calc_maxpool_output_size(
                seq_len=conv_out_size,
                kernel_size=self.maxpool_kernel_size,
                stride=self.maxpool_kernel_size,
                padding=self.maxpool_padding,
                dilation=self.maxpool_dilation
            )
            
            # Add output sizes
            self.flatten_size += maxpool_out_size
            
        # Compute final size of flattened input to fully connect layers
        self.flatten_size *= self.out_channels[-1]
        
        # Set fc layers with dropout
        self.fc0 = nn.Linear(self.flatten_size, n_fc_neurons_l0)
        self.drop0 = nn.Dropout(p=fc_dropout_rate_l0)
        self.fc1 = nn.Linear(n_fc_neurons_l0, n_fc_neurons_l1)
        self.drop1 = nn.Dropout(p=fc_dropout_rate_l1)
        self.output = nn.Linear(n_fc_neurons_l1, n_classes)
        
    def _calc_conv_output_size(self, seq_len, kernel_size, stride, padding):
        
        return int(((seq_len - kernel_size + 2 * padding) / stride) + 1)

    def _calc_maxpool_output_size(self, seq_len, kernel_size, stride, padding, dilation):
        
        return int(math.floor(((seq_len + 2 * padding - dilation * (kernel_size - 1) - 1) / stride ) + 1 ))

    def forward(self, input):
        
        # Get embeddings from input tokens
        # Output shape: (batch_size, max_seq_len, embedding_dim)
        x = self.embedding(input)
        
        # Permute embedding output to match input shape requirement of nn.Conv1d
        # Where embedding_dim = input channels
        # Output shape: (batch_size, embedding_dim, max_seq_len)
        x = x.transpose(1, 2)
        
        # Run through conv, dropout, and maxpool layers
        xs = []
        for i in range(0, len(self.conv_layers)):
            l_out = F.relu(self.conv_layers[f'conv_{i}'](x))
            l_out = self.dropout_layers[f'dropout_{i}'](l_out)
            l_out = self.maxpool_layers[f'maxpool_{i}'](l_out)
            l_out = l_out.view(l_out.size(0), -1)
            xs.append(l_out)
        
        # Concatenate conv output layers
        x = torch.cat(xs, 1)
        
        # Run through fully connected layers
        x = F.relu(self.fc0(x))
        x = self.drop0(x)
        x = F.relu(self.fc1(x))
        x = self.drop1(x)
        logits = self.output(x)
        
        # Convert to class probabilities
        probs = torch.sigmoid(logits)
        
        return probs

Two related follow-up questions:

In the code above, you’ll see where I define self.in_channels and self.out_channels that I note how I would like to increase the out_channels size at each layer.  I get errors like Given groups=1, weight of size [256, 256, 6], expected input[32, 128, 5000] to have 256 channels, but got 128 channels instead when I attempt to increase this.  Is there a way to increase the size of the out_channels?

@vdw, I see that you concatenate the output of each CNN + Maxpool operation in your code and pass these features to your fully connected layers.  Other PyTorch code I’ve seen will run the input data through each layer and pass the final output to the fully connected layers.  I assume these approaches are very different in that concatenation provides features from each layer representing various levels of abstraction, whereas passing the final layer would just provide the representation from the last layer.  Am I thinking about that right?