Predicting using ML model on training set If i train my model using 70% of the data, and then test on the remaining 30% ... I am getting 65% accuracy predicting on the test set.
Out of interest i predicted on the training set (the same data from which i trained the model). I get roughly the same accuracy score (65%) with Logistic Regression. With Random Forest i get 80% predicting on the training set and around 65% predicting on the test set. So, in summary, with Logistic Regression i get consistent number whether i predict on the training set or the test set. With Random Forest i get a higher number when i predict on the training set. The accuracy from both models predicting on the test set is the same.
I am trying to understand which model is doing better. Should predicting on your training set always throw out a very a high accuracy? Or would that indicate overfitting?
Can I learn something from these stated measures? Perhaps i can conclude that the Random Forest is overfitting because accuracy from predicting on the training set is so much higher accuracy than predicting on the test set (not withstanding predicting on the test set is giving a reasonable number)?
If the model generalises well should the accuracy of predicting on training set and test set, be roughly the same? Is that in itself a good test for "generalising well", assuming the accuracy is a decent number?
Any input gratefully received.


Should predicting on your training set always throw out a very a high accuracy? Or would that indicate overfitting?

No, not always.  The reason some models perform better than others is due to the bias/variance tradeoff.  Logistic regression is a linear model, and hence is high bias.  A random forest is capable of learning non-linear effects (low boas) of variables, but this comes at the cost of more variance.  This increase in accuracy is likely due to the lower bias of random forests.

Can I learn something from these stated measures?

Not particularly, the training error is a poor measure of performance since you're testing on things the algorithm has already seen.  The model has to fit the data as best as possible, so the training error is ultimately uninformative as to future performance.

If the model generalises well should the accuracy of predicting on training set and test set, be roughly the same?

Maybe.  Models can generalize well and still be slightly overfit.
