How to evaluate two nested neural network? I am currently working on a classification model based on tabular dataset (dataset size is much larger than the feature size) and performing some experiments on new features.
Here is some context about the problem statement:

*

*A and B are two nested shallow neural network such that A is trained on X features and B is trained on (X + x') features.

*Train/val/test set is same for both the models except the inclusion of feature x' for model B.

I wanted to understand if this method of selecting the best model is correct:
Train model A on trainA and perform validation using valA, similarly train model B using TrainB and perform validation using valB. Finally, test model A on testA and model B on test B and select the one with highest AUC score.
Is there any issue with the above technique? can you point me to a direction to handle model selection for such cases.
Thank you.

It's better if your train, validation and test sets are the same in terms of samples (not features of course) to compare the models. Because you're changing two things, the dataset and the model (feature selection actually).
