Do I want to overfit, when doing outlier detection based on regression? Imagine, we have speed data of car and we would like to detect, if car speeds up or down more than it should.
Do I want to just overfit my model, so the outlier (higher or lower speed) would lead me to higher residual easing the process of outlier detection?
Edit: To clarify, training data have no outliers.


Do I want to just overfit my model, so the outlier (higher or lower speed) would lead me to higher residual easing the process of outlier detection?

No, you don't. A heavily overfitted model would perfectly fit your data, but not generalize outside it. So it can be the case that it would consider as an outlier anything that is not alike to your training data. For an overfitted model to work for outlier detection, your training data would need to contain every possible scenario to be observed in the real world. But, if you had data on every possible scenario, you wouldn't need a machine learning model, since you could simply search your database and if the datapoint is not found among the "valid" cases, it is an anomaly. Most likely, your data does not contain every possible scenario, especially since speed is a real number so there is an infinite number of them. That is why you need a model that is able to fill in the gaps, so it needs to generalize outside of the training data.
