<p>Since you're using categorical cross-entropy loss, the last layer (output layer) should come with <code>softmax</code> activation instead of <code>identity</code> (as being blank in <code>layers.Dense(4)</code>).</p>
<pre><code>model = tf.keras.Sequential([
    normalize,
    ...,
    layers.Dense(4, activation=tf.keras.activations.softmax)
])
</code></pre>
<p>And <code>SparseCategoricalCrossentropy</code> is different from <code>CategoricalCrossentropy</code>, for categorical cross-entropy, the shape of output and label should be the same, for example:</p>
<p>When using categorical cross-entropy (example labels in one-hot):</p>
<pre><code>Output: [[.1,.0,.9,.0],...] &lt;-- Prediction of 90% for the 3rd class, sum=100%
Label:  [[ 0, 0, 1, 0],...] &lt;-- Third class holds 100% classification possibility
</code></pre>
<p>When using sparse categorical cross-entry (labels are class indices):</p>
<pre><code>Output: [[.1,.0,.9,.0],...] &lt;-- Prediction of 90% for the 3rd class, sum=100%
Label:  [[2]          ,...] &lt;-- Third class index is 2
</code></pre>

CC BY-SA 4.0
2021-02-19T01:44:40.717Z
2021-02-19T01:58:57.137Z
2021-02-19T01:58:57.137Z
Dan D.
<p>I am creating a NN in tensorflow keras. the inputs are all float and the output is a class.</p>
<p>The output currently encoded as a float, but only has 4 values (0,1,2,3).</p>
<p>My model is similar to this:</p>
<pre><code>model = tf.keras.Sequential([
    normalize,
    layers.Dense(128, activation='relu'),
    layers.Dense(254, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(254, activation='relu'),
    layers.Dense(128, activation='relu'),
    layers.Dense(4)
])

model.compile(loss = tf.keras.losses.SparseCategoricalCrossentropy(), metrics=['accuracy'],
                           optimizer = tf.optimizers.Adam())

history=model.fit(data_set_features, data_set_labels,validation_split=0.33, epochs=100)
</code></pre>
<p>is the model last layer correct?</p>
<p>what type of activation should I use and loss function?</p>

CC BY-SA 4.0
2021-02-19T10:32:24.913Z
Programming questions are off-topic here. Please, edit your post to clarify how this is not a programming question.
nbro
<tensorflow><keras><objective-functions><activation-functions>
setting up last layer in tensoflow for class type of label
mans
# setting up last layer in tensoflow for class type of label

(Asked by: username_0 on 2021-02-19)

I am creating a NN in tensorflow keras. the inputs are all float and the output is a class.
The output currently encoded as a float, but only has 4 values (0,1,2,3).
My model is similar to this:
model = tf.keras.Sequential([
    normalize,
    layers.Dense(128, activation='relu'),
    layers.Dense(254, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(254, activation='relu'),
    layers.Dense(128, activation='relu'),
    layers.Dense(4)
])

model.compile(loss = tf.keras.losses.SparseCategoricalCrossentropy(), metrics=['accuracy'],
                           optimizer = tf.optimizers.Adam())

history=model.fit(data_set_features, data_set_labels,validation_split=0.33, epochs=100)

is the model last layer correct?
what type of activation should I use and loss function?

**Tags:** tensorflow, keras, objective-functions, activation-functions

> **Comments:**
> **username_1**: Programming questions are off-topic here. Please, edit your post to clarify how this is not a programming question.  

---

## 1 answers

### Answer by username_2 on 2021-02-19. Score: 1 | Accepted answer

Since you're using categorical cross-entropy loss, the last layer (output layer) should come with softmax activation instead of identity (as being blank in layers.Dense(4)).
model = tf.keras.Sequential([
    normalize,
    ...,
    layers.Dense(4, activation=tf.keras.activations.softmax)
])

And SparseCategoricalCrossentropy is different from CategoricalCrossentropy, for categorical cross-entropy, the shape of output and label should be the same, for example:
When using categorical cross-entropy (example labels in one-hot):
Output: [[.1,.0,.9,.0],...] <-- Prediction of 90% for the 3rd class, sum=100%
Label:  [[ 0, 0, 1, 0],...] <-- Third class holds 100% classification possibility

When using sparse categorical cross-entry (labels are class indices):
Output: [[.1,.0,.9,.0],...] <-- Prediction of 90% for the 3rd class, sum=100%
Label:  [[2]          ,...] <-- Third class index is 2


---

