Why would a Linear SVM perform worse than Logistic Regression? I have a dataset of about 1000 features and 2000 training examples. I use Randomized Search with cross validation to compare Random Forest, Linear SVC, SVC with different non-linear kernels, and Logistic Regression.
I consistently get much worse scores with Linear SVC than with the other models, including Logistic regression (AUC ROC of 0.70 for Linear SVC vs 0.80 for other models).
From what I understand, the performance of the Linear SVMs should be comparable to Logistic Regression. What could be the reason for such poor performance?
Here are the hyperparamaters I'm checking:
rf_param_grid = {
    'randomforestclassifier__max_depth' : np.random.randint(5, 150, 30),
    'randomforestclassifier__min_samples_split': np.random.randint(2, 50, 30),
    'randomforestclassifier__n_estimators': np.random.randint(50, 400, 10),
    'randomforestclassifier__min_samples_leaf': np.random.randint(1, 20, 30),
    'randomforestclassifier__max_features': ['auto', 'sqrt', 'log2', 0.25, 0.5, 0.75, 1.0],
    'randomforestclassifier__criterion': ['gini', 'entropy'],
    'randomforestclassifier__class_weight':["balanced", "balanced_subsample", None],
    "randomforestclassifier__class_weight": ['balanced', None]
}
linear_svc_param_grid = {
    'svc__C': [0.1, 0.2, 0.3, 0.4, 0.5, 1, 5, 10],
    "svc__class_weight": ['balanced', None]
}
kernel_svc_param_grid = {
    'svc__C': loguniform(1e-1, 1e3),
    'svc__gamma': loguniform(1e-04, 1e+01),
    'svc__degree': uniform(2, 5),
    'svc__kernel': ['poly', 'rbf', 'sigmoid'],
    "svc__class_weight": ['balanced', None]
}
lr_param_grid = {
    'logisticregression__C': loguniform(1e-5, 1e4),
    'logisticregression__penalty': ['l1', 'l2', 'elasticnet'],
    'logisticregression__class_weight': ['balanced', None],
    'logisticregression__l1_ratio': uniform(0, 1)
}


In addition to @sycorax's answer (+1), another issue is that the SVM is not designed for AUROC maximisation - it is designed to estimate the optimal decision surface for a particular set of misclassification costs, determined by the values of $C$ for each class (typically only a single $C$ value is used, in which case the misclassification costs are equal).  How it performs for other sets of misclassification costs (which is what ROC analysis is probing), is not of primary importance.
The logistic regression model on the other hand, aims to estimate the posterior probability of class membership, and rather than concentrating on a single decision threshold (usually $p=0.5$), it tries to estimate the posterior probabilities accurately everywhere.  This means that different misclassification costs can be accommodated just by changing the threshold value.  A consequence of this, there is more reason to expect that logistic regression will quite good at maximising ROC (at least more so than the SVM).
So if AUROC is the primary performance index, it probably isn't a suitable application for the SVM.
