If class indices [16, 20] are representing different classes than [0, 15], your classifier should output logits for 21 classes ([0, 20]). Otherwise, if the class indices are representing the same class but are for some reason shifted in their value, you should remap them.