Answers.Body: <p>Since you're using categorical cross-entropy loss, the last layer (output layer) should come with <code>softmax</code> activation instead of <code>identity</code> (as being blank in <code>layers.Dense(4)</code>).</p>
  <pre><code>model = tf.keras.Sequential([
      normalize,
      ...,
      layers.Dense(4, activation=tf.keras.activations.softmax)
  ])
  </code></pre>
  <p>And <code>SparseCategoricalCrossentropy</code> is different from <code>CategoricalCrossentropy</code>, for categorical cross-entropy, the shape of output and label should be the same, for example:</p>
  <p>When using categorical cross-entropy (example labels in one-hot):</p>
  <pre><code>Output: [[.1,.0,.9,.0],...] &lt;-- Prediction of 90% for the 3rd class, sum=100%
  Label:  [[ 0, 0, 1, 0],...] &lt;-- Third class holds 100% classification possibility
  </code></pre>
  <p>When using sparse categorical cross-entry (labels are class indices):</p>
  <pre><code>Output: [[.1,.0,.9,.0],...] &lt;-- Prediction of 90% for the 3rd class, sum=100%
  Label:  [[2]          ,...] &lt;-- Third class index is 2
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-02-19T01:44:40.717Z
Answers.LastActivityDate: 2021-02-19T01:58:57.137Z
Answers.LastEditDate: 2021-02-19T01:58:57.137Z
Answers.UserDisplayName: Dan D.
Body: <p>I am creating a NN in tensorflow keras. the inputs are all float and the output is a class.</p>
  <p>The output currently encoded as a float, but only has 4 values (0,1,2,3).</p>
  <p>My model is similar to this:</p>
  <pre><code>model = tf.keras.Sequential([
      normalize,
      layers.Dense(128, activation='relu'),
      layers.Dense(254, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(254, activation='relu'),
      layers.Dense(128, activation='relu'),
      layers.Dense(4)
  ])
  
  model.compile(loss = tf.keras.losses.SparseCategoricalCrossentropy(), metrics=['accuracy'],
                             optimizer = tf.optimizers.Adam())
  
  history=model.fit(data_set_features, data_set_labels,validation_split=0.33, epochs=100)
  </code></pre>
  <p>is the model last layer correct?</p>
  <p>what type of activation should I use and loss function?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-02-19T10:32:24.913Z
Comments.Text: Programming questions are off-topic here. Please, edit your post to clarify how this is not a programming question.
Comments.UserDisplayName: nbro
Tags: <tensorflow><keras><objective-functions><activation-functions>
Title: setting up last layer in tensoflow for class type of label
UserDisplayName: mans
ThreadText: # setting up last layer in tensoflow for class type of label
  
  (Asked by: username_0 on 2021-02-19)
  
  I am creating a NN in tensorflow keras. the inputs are all float and the output is a class.
  
  The output currently encoded as a float, but only has 4 values (0,1,2,3).
  
  My model is similar to this:
  
  ```
  model = tf.keras.Sequential([
      normalize,
      layers.Dense(128, activation='relu'),
      layers.Dense(254, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(512, activation='relu'),
      layers.Dense(254, activation='relu'),
      layers.Dense(128, activation='relu'),
      layers.Dense(4)
  ])
  
  model.compile(loss = tf.keras.losses.SparseCategoricalCrossentropy(), metrics=['accuracy'],
                             optimizer = tf.optimizers.Adam())
  
  history=model.fit(data_set_features, data_set_labels,validation_split=0.33, epochs=100)
  
  ```
  
  is the model last layer correct?
  
  what type of activation should I use and loss function?
  
  **Tags:** tensorflow, keras, objective-functions, activation-functions
  
  > **Comments:**
  > **username_1**: Programming questions are off-topic here. Please, edit your post to clarify how this is not a programming question.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-02-19. Score: 1 | Accepted answer
  
  Since you're using categorical cross-entropy loss, the last layer (output layer) should come with `softmax` activation instead of `identity` (as being blank in `layers.Dense(4)`).
  
  ```
  model = tf.keras.Sequential([
      normalize,
      ...,
      layers.Dense(4, activation=tf.keras.activations.softmax)
  ])
  
  ```
  
  And `SparseCategoricalCrossentropy` is different from `CategoricalCrossentropy`, for categorical cross-entropy, the shape of output and label should be the same, for example:
  
  When using categorical cross-entropy (example labels in one-hot):
  
  ```
  Output: [[.1,.0,.9,.0],...] <-- Prediction of 90% for the 3rd class, sum=100%
  Label:  [[ 0, 0, 1, 0],...] <-- Third class holds 100% classification possibility
  
  ```
  
  When using sparse categorical cross-entry (labels are class indices):
  
  ```
  Output: [[.1,.0,.9,.0],...] <-- Prediction of 90% for the 3rd class, sum=100%
  Label:  [[2]          ,...] <-- Third class index is 2
  
  ```
  
  
  ---
  
  