Thanks for the response,
The results provided remain the same if I don’t use any explicit initialization and remove the nn.Softmax() from the forward pass. This second attempt was to replicate the paper specs to see if this particular initialization was required for this particular network architecture.
Question
Even if the model does not have the “enough capacity” (16 conv layers with 32 filters each and a total of 170K parameters) to reach a 90% accuracy, just because of gradient descent we should see a reduction on the loss, at least until a certain point right?