Answers.Body: <p>Firstly, you should use sigmoid in your last layer instead of softmax. Softmax returns a probability distribution, meaning that when one labels probability increases the other will decrease, which is not always the case. Secondly, in order for Keras to use all the labels, you should use the binary cross-entropy as the loss function.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-26T10:44:14.027Z
Answers.Text: Thanks! I actually already used Sigmoid, just copied the wrong parts here, and also binary_crossentropy (following on Francois Chollet's book on Keras). This means it's working as intended.
Answers.UserDisplayName: Joseph Doob
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-26T09:30:15.683Z
Answers.LastActivityDate: 2022-04-08T20:56:26.027Z
Answers.LastEditDate: 2022-04-08T20:56:26.027Z
Answers.UserDisplayName: razvanc92
Body: <p>I have data with about 100 numerical features and a multi-labelling that encodes ownership of a certain product (i.e. my labels are of the form <span class="math-container">$[x_i, i=1, \dots, n]$</span>, where <span class="math-container">$n$</span> is the number of products and <span class="math-container">$x_i$</span> is either 0 or 1).</p>
  <p>My neural network approach to this currently looks like this (in Keras)</p>
  <pre><code>model = Sequential()
  model.add(Dense(1024, activation='relu', input_dim=X_train.shape[1]))
  model.add(Dense(1024, activation='relu'))
  model.add(Dense(1024, activation='relu'))
  model.add(Dense(1024, activation='relu'))
  model.add(Dense(y_train.shape[1], activation='softmax'))
  </code></pre>
  <p>So, it has a couple of dense layers with ReLu activation, then an output layer with softmax.</p>
  <p>Now, my question is: will the neural network consider labels of the other products when assigning a probability to the label of one product?</p>
  <p>I would like that to happen, but I can't quite grasp whether it does (my suspicion is no).</p>
  <p>I'm new to multi-label classification and relatively new to NN in general, so I hope this isn't too inept a question.</p>
  
Tags: <neural-networks><keras><objective-functions><softmax><multi-label-classification>
Title: Is this neural network with a softmax in the output layer suitable for multi-label classification?
UserDisplayName: Joseph Doob
ThreadText: # Is this neural network with a softmax in the output layer suitable for multi-label classification?
  
  (Asked by: username_0 on 2019-04-26)
  
  I have data with about 100 numerical features and a multi-labelling that encodes ownership of a certain product (i.e. my labels are of the form $[x\_i, i=1, \dots, n]$, where $n$ is the number of products and $x\_i$ is either 0 or 1).
  
  My neural network approach to this currently looks like this (in Keras)
  
  ```
  model = Sequential()
  model.add(Dense(1024, activation='relu', input_dim=X_train.shape[1]))
  model.add(Dense(1024, activation='relu'))
  model.add(Dense(1024, activation='relu'))
  model.add(Dense(1024, activation='relu'))
  model.add(Dense(y_train.shape[1], activation='softmax'))
  
  ```
  
  So, it has a couple of dense layers with ReLu activation, then an output layer with softmax.
  
  Now, my question is: will the neural network consider labels of the other products when assigning a probability to the label of one product?
  
  I would like that to happen, but I can't quite grasp whether it does (my suspicion is no).
  
  I'm new to multi-label classification and relatively new to NN in general, so I hope this isn't too inept a question.
  
  **Tags:** neural-networks, keras, objective-functions, softmax, multi-label-classification
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-04-26. Score: 1 | Accepted answer
  
  Firstly, you should use sigmoid in your last layer instead of softmax. Softmax returns a probability distribution, meaning that when one labels probability increases the other will decrease, which is not always the case. Secondly, in order for Keras to use all the labels, you should use the binary cross-entropy as the loss function.
  
  > **Comments:**
  > **username_0**: Thanks! I actually already used Sigmoid, just copied the wrong parts here, and also binary_crossentropy (following on Francois Chollet's book on Keras). This means it's working as intended.  
  
  ---
  
  