First, you should treat age as a regression problem unless there is a good reason you need to do classification.
Second, you can simply index into the output of your models and select the rows where you have an output, then calculate the loss only on those rows.