Solved by ptrblck in post #5 
              
              
                No, your last layer should return the number of classes. So it would have dimensions [batch_size, classes]. The target vector should only contain the class indices.