When to Normalize Training Data As I have seen so far, it seems like choices such as whether or not to normalize your training data are made based on the results you get after evaluating your model on the test data. Is there a more general a priori explanation for when data normalization is appropriate?

If the normalization decision is made based on the test data, it'll be a biased estimate for the model's generalization performance. This decision should be done based on the validation set(s). The normalization is usually appropriate but it's more essential if the variables have wildly different ranges and the algorithm used is sensitive to them, e.g. ridge regression. Or, when you have regularization on the weights multiplied with the input features. It may also badly affect gradient descent performance (in algorithms like neural networks), and the ranges outputted by functions like sigmoid and tanh. On the other hand, tree based algorithms usually does not require normalization because what you're looking for are only split points, which won't change with normalization. So, it depends on the algorithms, whether you have regularization or not and its type, and whether you'd like to treat features fairly or not (as usually done in PCA).
