Answers.Body: <blockquote>
  <p>Let's say we have <span class="math-container">$K$</span> classes. For <span class="math-container">$K$</span> classes, we will be training <span class="math-container">$K$</span> different neural networks.</p>
  </blockquote>
  <p>No, you still train <strong>one</strong> network.</p>
  <p>With binary classification tasks, where you have only two mutually exclusive categories, like &quot;yes/no&quot; or &quot;true/false&quot;, you can get away with a single output node with a sigmoid activation. The output of the sigmoid is interpreted as indicating one category for values <span class="math-container">$&gt; 0.5$</span> and the other for values <span class="math-container">$\leq 0.5$</span>.</p>
  <p>With multi-class classification, you have <span class="math-container">$K$</span> outputs (one for each category). The problem, in this case, is that if the network gets the class wrong, in general, you cannot decide in one step which one of the other <span class="math-container">$K - 1$</span> categories is the correct one. So, the output is actually passed through an extra <a href="https://en.wikipedia.org/wiki/Softmax_function" rel="nofollow noreferrer">softmax</a> layer, which outputs <em>probabilities</em> for each class.</p>
  <blockquote>
  <p>But do we train one neural network at a time for all features, or do we train all <span class="math-container">$K$</span> neural networks at a time for one feature?</p>
  </blockquote>
  <p>You present <em>all</em> features for each training example to the network at the same time. So, for <span class="math-container">$N$</span> features you have <span class="math-container">$N$</span> input nodes, and you feed <em>all</em> of them into the neural network.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-08T12:01:08.803Z
Answers.Text: okay it means for K classes and M training examples we will be training our neural network K*M times.
Answers.UserDisplayName: Reena Kandari
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-08T12:08:55.473Z
Answers.Text: One more doubt sir.Do we apply gradient descent(any varient) when using neural network? in [this](https://github.com/m-a-y-a-n-k/Weather-Prediction-Using-Neural-Networks/blob/master/Predictor.m) code they are applying advance optimization before Back Propagation. According to me in BP as we are already finding the best set of weights then applying gradient descent is irrelevant in this case.Am I correct or  missing something ?
Answers.UserDisplayName: Reena Kandari
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-09T22:32:28.130Z
Answers.Text: @ReenaKandari If you have M data points, you would be using a subset (generally, between `60%` and `80%` of `M`) for training, and you do that multiple times (= epochs). However, you don't have to train the network `K` times for each training example - the whole point of softmax is that the correct class is "promoted" while *all* others are "suppressed" *simultaneously*, if that makes sense.
Answers.UserDisplayName: cantordust
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-09T22:33:44.283Z
Answers.Text: @ReenaKandari Re your other question: yes, in the overwhelming majority of cases neural networks are optimised using gradient descent. I am not proficient in Matlab, but it seems to me that in the example you provided they are minimising a cost function using gradient descent.
Answers.UserDisplayName: cantordust
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-14T02:32:15.887Z
Answers.Text: sir @cantordust, actually I am not aware of how Softmax works and how classes are 'promoted' and 'suppressed simultaneously '.Can you please explain it a little bit more that how these things  works in terms of neural network?
Answers.UserDisplayName: Reena Kandari
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-07T23:36:33.663Z
Answers.LastActivityDate: 2021-07-15T13:27:20.283Z
Answers.LastEditDate: 2021-07-15T13:27:20.283Z
Answers.UserDisplayName: cantordust
Answers.Body: <p>Let us suppose that you are training a neural network for classfing images of vehicles , then the input vector , image of the "vehicle" will be a 2D array of pixels. This undergoes several transformations at each layer of the neural network , the last layer of the neural network produces another vector whose dimensions are lesser than the original image vector.</p>
  
  <p>So the network is mapping images to some vectors in a high dimensional space. In order to classify the images it is now sufficient to classify the vectors obtained from the network of their corresponding images, you  can do this with a simple "linear" classifier using a softmax layer.</p>
  
  <p>So all the layers of the network except the last layer are transforming the image representation to a "vector". This vector is classified by a linear softmax classifier by the last layer of neural network.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-07-07T06:30:40.930Z
Answers.LastActivityDate: 2018-07-07T06:30:40.930Z
Answers.UserDisplayName: thecomplexitytheorist
Body: <p>I am very new to machine learning. I am following the course offered by Andrew Ng. I am very confused about how we train our neural network for multi-class classification.</p>
  <p>Let's say we have <span class="math-container">$K$</span> classes. For <span class="math-container">$K$</span> classes, we will be training <span class="math-container">$K$</span> different neural networks.</p>
  <p>But do we train one neural network at a time for all features, or do we train all <span class="math-container">$K$</span> neural networks at a time for one feature?</p>
  <p>Please, explain the complete procedures.</p>
  
Tags: <neural-networks><multiclass-classification>
Title: What is the general procedure to use and train neural networks for multi-class classification?
UserDisplayName: Reena Kandari
ThreadText: # What is the general procedure to use and train neural networks for multi-class classification?
  
  (Asked by: username_0 on 2018-07-07)
  
  I am very new to machine learning. I am following the course offered by Andrew Ng. I am very confused about how we train our neural network for multi-class classification.
  
  Let's say we have $K$ classes. For $K$ classes, we will be training $K$ different neural networks.
  
  But do we train one neural network at a time for all features, or do we train all $K$ neural networks at a time for one feature?
  
  Please, explain the complete procedures.
  
  **Tags:** neural-networks, multiclass-classification
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-07-07. Score: 2 | Accepted answer
  
  > Let's say we have $K$ classes. For $K$ classes, we will be training $K$ different neural networks.
  
  No, you still train **one** network.
  
  With binary classification tasks, where you have only two mutually exclusive categories, like "yes/no" or "true/false", you can get away with a single output node with a sigmoid activation. The output of the sigmoid is interpreted as indicating one category for values $> 0.5$ and the other for values $\leq 0.5$.
  
  With multi-class classification, you have $K$ outputs (one for each category). The problem, in this case, is that if the network gets the class wrong, in general, you cannot decide in one step which one of the other $K - 1$ categories is the correct one. So, the output is actually passed through an extra [softmax](https://en.wikipedia.org/wiki/Softmax_function) layer, which outputs *probabilities* for each class.
  
  > But do we train one neural network at a time for all features, or do we train all $K$ neural networks at a time for one feature?
  
  You present *all* features for each training example to the network at the same time. So, for $N$ features you have $N$ input nodes, and you feed *all* of them into the neural network.
  
  > **Comments:**
  > **username_0**: okay it means for K classes and M training examples we will be training our neural network K*M times.  
  > **username_0**: One more doubt sir.Do we apply gradient descent(any varient) when using neural network? in [this](https://github.com/m-a-y-a-n-k/Weather-Prediction-Using-Neural-Networks/blob/master/Predictor.m) code they are applying advance optimization before Back Propagation. According to me in BP as we are already finding the best set of weights then applying gradient descent is irrelevant in this case.Am I correct or  missing something ?  
  > **username_1**: @username_0 If you have M data points, you would be using a subset (generally, between `60%` and `80%` of `M`) for training, and you do that multiple times (= epochs). However, you don't have to train the network `K` times for each training example - the whole point of softmax is that the correct class is "promoted" while *all* others are "suppressed" *simultaneously*, if that makes sense.  
  > **username_1**: @username_0 Re your other question: yes, in the overwhelming majority of cases neural networks are optimised using gradient descent. I am not proficient in Matlab, but it seems to me that in the example you provided they are minimising a cost function using gradient descent.  
  > **username_0**: sir @username_1, actually I am not aware of how Softmax works and how classes are 'promoted' and 'suppressed simultaneously '.Can you please explain it a little bit more that how these things  works in terms of neural network?  
  
  ---
  
  ### Answer by username_2 on 2018-07-07. Score: 2
  
  Let us suppose that you are training a neural network for classfing images of vehicles , then the input vector , image of the "vehicle" will be a 2D array of pixels. This undergoes several transformations at each layer of the neural network , the last layer of the neural network produces another vector whose dimensions are lesser than the original image vector.
  
  So the network is mapping images to some vectors in a high dimensional space. In order to classify the images it is now sufficient to classify the vectors obtained from the network of their corresponding images, you can do this with a simple "linear" classifier using a softmax layer.
  
  So all the layers of the network except the last layer are transforming the image representation to a "vector". This vector is classified by a linear softmax classifier by the last layer of neural network.
  
  
  ---
  
  