Are you sure you are passing the images of correct size during evaluation?  It seems there could be a mistake in that part. Can you please post your code to have a look? It’s difficult to hypothesize without code.