Sklearn uses a coordinate descent algorithm. An exact, sparse solution is reached after a finite number of iterations. No stochasticity or SGD involved. Fairly different framework from back-propagation.
This is something I am potentially interested in contributing, but for now I’d like to understand the added value of having Lasso-like solvers in PyTorch and if they could fit with the existing pytorch optimizers.