This is an old topic, but it was still relevant to a project I’m working on, so I’ll share what I found while investigating:
There are three main differences between typical scikit-learn and pytorch implementations of logistic regression: the regularization, the data handling, and the optimizer.
ptrblck already touched on regularization difference, namely that scikit-learn used L2 weight regularization by default, while pytorch models are unregularized by default.
The data handling doesn’t make much of a performance difference from what I’ve seen, but is worth mentioning. SKL models call fit on the entire training set, while most pytorch training loops use mini-batch gradient descent where they update on a fraction of the data at a time.
The most important factor (at least for my problem) was the difference between optimizers. SKL defaults to LBFGS, a second-order method, while pytorch models typically use Adam. I found that I was able to get better performance out of my logistic regression model by using the pytorch LBFGS optimizer 1. It failed to reach the same accuracy as an unregularized scikit-learn logistic regression though, so there may still be more to the story.