How does a gradient pass through argmax in classification? I just realized I have not given this issue much thought. In a classification task, there is an argmax happening after the softmax to get the most likely class. So how does backpropagation go through that operation?

It does not. Almost always the model is trained to predict real-valued score (e.g. probability) by minimizing some smooth loss function. While training such a model never makes hard classifications, so never use argmax. The final layer of most of the classification neural networks is softmax rather than hardmax. You can make the hard classification only after the training, by using the score returned by the model.
See also the related thread Why is accuracy not the best measure for assessing classification models? that discusses problems with accuracy as a metric, but many of the points apply as well to other non-smooth metrics. Training the model to minimize such metric would actually be harder.
