swarup:

nn.Linear(4096, 1024),


Your first linear layer has input number of features = 4096.



 swarup:

X = self.conv_layer(X)
    
# flatten
X = X.view(X.size(0), -1)



You have to make sure that after doing above quoted operations the number of features of X (X.size(1)) is also equal to 4096.
In general, you have to adapt the number of input features of the first linear layer to the number of output features after the last convolution operation, which in your case is:



 swarup:

        nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=2, stride=2),