edshkim98:

and instead should aim to increase the validation dataset accuracy?


That’s hard to tell, since your validation dataset is not a good proxy of the test data.
I.e. even if your training and validation performance is great, your model might just fail on the test dataset, since the distribution could be too different.
Note that “looking” into the test set is not a good idea (you should not observe the test predictions during training), as you would leak the test information into the training.
One other approach, which comes to my mind, would be to use a small portion of the test set to train the model in an unsupervised fashion, e.g. to reconstruct the input.
For this you could use a special “branch” for the output layers to make sure the output shape fits the input shape and remove it later during the classification training. This could maybe pretrain some early layers, but you would have to experiment with it.