How do I know my model overfits? We already have multiple questions on overfitting, however, we also observe certain kinds of questions to re-occur regularly that ask how to diagnose overfitting in practice. Let's try giving an authoritative answer for such a question, so it could serve as a reference.
How do I know my model overfits? For example: does perfect training error mean overfitting, or how big does the gap between training and test error need to be to call it overfitting?

While this question is prone to soliciting "takes" rather than answers, here's mine:
You don't.
Assuming you have used all available resources (i.e. data), regardless of the methodology applied, you don't know if your model is overfit or not. Conversely, modeling techniques that are prone to bias and overfitting may be accurate after all. During independent review, models built using poor methods are dismissed because those methods have poor operating characteristics, but even the crappy archer hits the occasional bullseye. All this is true inspite of any pre-specified objectives, or knowledge of what "plausible" or "good" models might look like apriori. For instance, before the BRCA gene was identified, we had a somewhat pessimistic view of how accurate breast cancer risk models could be, and nothing could have anticipated how powerful their prognostic value might be based on previously researched risk factors.
