Body: <p>For a project, I've trained multiple networks for multiclass classification all ending with a ReLU activation at the output.</p>
  <p>Now the output logits are not probabilities.</p>
  <p>Is it valid to get the probability of each class by applying a softmax function at the end, after training?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-22T12:08:13.847Z
Comments.Text: Hello. Welcome to Artificial Intelligence Stack Exchange! Can you please explain 1. why you trained more than one neural network to solve the same problem, 2. why you used the ReLU at the last layers of these neural nets, 3. how exactly you trained the neural nets? Did you compare the (vector) labels with the output produced by the last ReLU, 3.1 you got good accuracy with this approach?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-22T12:23:16.700Z
Comments.Text: Hi, thank you.
  1. It's not the exact same problem, it's for slightly differing datasets and slight differing architectures.
  2. This is a constraint set for research purposes.
  3. As usual, using backprop - output decision was decided using argmax over the last relu output vector; and yes test/validation accuracy and loss are satisfying
Comments.UserDisplayName: user452306
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-22T12:24:43.513Z
Comments.Text: My main and only question, is if we can convert post-hoc some relu-output to some 'propabilities' in a valid way
Comments.UserDisplayName: user452306
Tags: <neural-networks><probability><softmax><multiclass-classification>
Title: Use soft-max post-training for a ReLU trained network?
UserDisplayName: user452306
ThreadText: # Use soft-max post-training for a ReLU trained network?
  
  (Asked by: username_0 on 2021-11-22)
  
  For a project, I've trained multiple networks for multiclass classification all ending with a ReLU activation at the output.
  
  Now the output logits are not probabilities.
  
  Is it valid to get the probability of each class by applying a softmax function at the end, after training?
  
  **Tags:** neural-networks, probability, softmax, multiclass-classification
  > **Comments:**
  > **username_1**: Hello. Welcome to Artificial Intelligence Stack Exchange! Can you please explain 1. why you trained more than one neural network to solve the same problem, 2. why you used the ReLU at the last layers of these neural nets, 3. how exactly you trained the neural nets? Did you compare the (vector) labels with the output produced by the last ReLU, 3.1 you got good accuracy with this approach?  
  > **username_0**: Hi, thank you. 1. It's not the exact same problem, it's for slightly differing datasets and slight differing architectures. 2. This is a constraint set for research purposes. 3. As usual, using backprop - output decision was decided using argmax over the last relu output vector; and yes test/validation accuracy and loss are satisfying  
  > **username_0**: My main and only question, is if we can convert post-hoc some relu-output to some 'propabilities' in a valid way  
  
  ---
  
  ## 0 answers
  
  