Solved by ptrblck in post #4 
              
              
                OK, in that case the second approach would be valid. 
If you have one valid class for each sample, your target should have the shape [batch_size] storing the class index. E.g. if the current word would be class5, you shouldn’t store it as [[0, 0, 0, 0, 0, 1, 0, ...]], but rather just use the class i…