There are a couple of things here.


First of all, the batch-size should not be given as input size to the Linear layer. The input size is independent of batch-size.


the linear cannot be placed in the same Sequential function right after the conv layers. The reason is the input to linear needs to be reshaped. So you have to create two sections, one Sequntial for all the conv layers (and their pooling, and activations, …), and then the output of this is reshaped to a flat tensor of shape batch-size x ?, and then this is passed to the linear layers.


    self.conv_layers = nn.Sequential(
        ## put the conv layers here
    )

   self.fc_layers = nn.Sequential(
       ## put the FC layers here
   )

   def forward(self, x):
      x = self.conv_layers(x)
      x = x.view(-1, ??) ## we need to know the value of ?? for the input size to the first linear layer
      return self.fc_layers(x)