Thanks for the update.
Based on your description you are missing some categorical features during evaluation.
Replacing them with a zero or invalid value might yield bad performance or invalid outputs (e.g. NaNs).
You could try to replace the missing values with the mean or median from your training set for this particular feature.