Is it normal for simple logistic regression to significantly outperform any other statistical ML algorithm? I'm working on a simple classification project with an imbalanced (minority-to-majority-ratio ~ 0.2) dataset that has ~4000 rows and ~200 features.
I noticed that, for my dataset, a simple logistic regression significantly outperforms most other classification algorithms. The ROC AUC score for the validation data in my LR model is ~0.8, compare to 0.52-0.62 for other algorithms. I tried many different algorithms such as RF, GBM, XGBoost, LighGBM, SVM, etc. and used SkOpt's Bayesian optimization to tune the hyperparameter in each algorithm.
I'm trying to understand what intrinsically is different about my data and was wondering if anyone has encountered such superior performance from LR and what were their thoughts.

My informal answer is that maximum likelihood estimation, the method behind logistic regression, finds the set of parameters that fit the data the best given some assumptions. If your dataset satisfies those assumptions very well and you have lots of data, then it is difficult to do better.
This paper about logistic regression vs random forest (I just found the paper by Googling) reports that RF performed better than LR according to the considered accuracy measured in approximately 69% of the datasets. So it suggests that is not unusual for logistic regression to beat random forest.
Also, Wikipedia reports that on the MNIST dataset, the linear classifier has error rate of 7.6% which is higher than other methods but I would say it is pretty good in absolute terms.
My impression is that older techniques like logistic regression are a bit underrated relative to modern ones like random forest or SVM but in many cases they are still preferable. My 2p-
