I have confirmed that the dimensions of images in my data generators were flipped. It appears that the fit() and evaluate() functions will automatically rotate images to fit the input of a model for you, however calling the model directly on an input will not. After fixing the order of my dimensions and retraining, calling the model directly gives me the same accuracy as using evaluate(). Thank you, everyone, for your help.