What are the best ML models for hundreds of binary features (but a non-binary response variable)? I have hundreds of binary features, resulting in a large binary design matrix (though note that my response variable is not binary). I've tried typical models like logistic regression, KNN, and SVMs with specialized kernels (like the Hamming kernel, mentioned here). I've also tried reducing the dimensionality of the data using PCA, though whether PCA is valid for binary data is debated (for example here and here).
None of these approaches have got me very far. Are there other models, dimensionality reduction techniques, feature engineering/selection techniques, etc that are well suited to a problem with hundreds of binary features?
Edit: these binary features were produced from one-hot encoding categorical features with many categories. I first tried encoding these features with integers but that didn't get me far either, probably because the categories are not ordinal.

I have found that non-negative-matrix-factorization is very useful for large numbers of binary features. In my experience, it tends to create a small number of dense feature vectors that are interpretable as plausible combinations of one-hot features that tend to go together. But this assumes that your hundreds of binary columns are the result of using one-hot or dummy encoding for several categorical variables.
Entity embeddings could also be useful, if you (1) want to use a neural network and (2) have several high-cardinality categorical features to encode.
If all of your columns encode a single categorical variable, then there's really not much to do: most models will just estimate a constant for each level of the category. (A corner case here is a model that uses shallow decision trees. Shallow trees won't be able to split the categories to purity, so some categories will be grouped together.)
