With the dropout, the loss was barely declining, even with thousands of epochs. I used a dropout of 0.3, but between the second and third convolutional layers. I can go back and try  a dropout between the linear layers instead.
Meanwhile, I’m currently doing augmentation, manually, to double the number of training samples.