@vdw, The suggestion worked
I used the Udacity’s Notebook to change the output and final activations in my model. The outputs were changed from (batch_size, seq_len, num_classes) to (batch_size, seq_len), after I removed OHEncoding for the output. During the training time, the raw logit outputs were used in the loss function and in the inference stage I applied Softmax to change the model outputs to probabilities. I guess that’s the only way to do it.