Solved by ptrblck in post #2 
              
              
                The 16 classes should be mapped to the range [0, 15], while your target uses sparse values between 0 and 45. 
You would have to remap these target values, as they are used to index the model output to calculate the loss.