A General question in mind. If I was training a classification model that I would then want to use for inference, wouldn’t it always be preferable to have a softmax layer at the end, instead of a regular linear that is input into a cross entropy loss function?
Wouldn’t the former make it easier to understand NN outputs, and also make it easier during inference, since all you need is a forward call and nothing else, unlike in the latter case where you need to process the NN output with some code?