I try to answer all questions, but take some of them with a grain of salt, as some of them are my personal opinion.


You would have to create a test dataset in the same manner as the train and val datasets were created.
Also, you could create a new transformation for the test Dataset or just use the val transform.
In train_model you should change the phase loop to for phase in ['train', 'val', 'test']:. The code should work fine on the test data, as the optimization conditions are performed on train, such that no learning takes place for val and test.


You could create a confusion matrix 3 and fill it with the preds and targets during val and test. After all samples were added to the confusion matrix, you can calculate the per-class accuracy by dividing by the column (or row) sums (that depends on your confusion matrix layout). If you don’t feel like writing the code yourself, sklearn 2 provides an implementation.


Usually you would have to report the test accuracy, as the validation data is “dirty”, i.e. it was usually used for hyperparameter tuning. For example if you observe the validation loss for early stopping or to alter your model architecture, you create a data leak, which marks the val data as “seen” by the model. That’s also the reason why your model shouldn’t see the test data until you’re finished with your experiment. Whether everyone sticks to it is a different matter.


Use the number of epochs, which works the best for your dataset. It’s hard to tell how many epochs your model needs in training. Especially if you’re using something like cyclic learning rate, you might need way less epochs to get to the same accuracy as other training strategies.


That’s also a bit hard to tell, as it depends on your data and use case. While ResNet101 has more layers, therefore potentially more capacity, it might be easier to overfit to your data so that your validation/test accuracy is worse. Try both architectures on your validation data and chose the one which is working better.