This was solved using a lower learning rate. The initial learning rate was 0.05, while I changed it to 0.005.
My question now is: Why did lowering the learning rate solve the problem in this case? In the other issue I had (the issue you linked), I had a deeper model (ResNet 50; in the book they used ResNet 18), so lowering the learning rate made sense as it lowered the probability of vanishing or exploding gradients. Here, however, I used ResNet 18, as they do in the book. I just changed the order of operations in the residual blocks.
Do you have any idea as to why changing the learning rate worked in this case? Again, here the model is not deeper. The only thing that changed is the order of the operations in the residual layer.