Pytorch lets you choose which optimization algorithm to use for any penalty that you would like to optimize.Here’s an example that uses SGD to optimize the L1 penalty. link 199
You could replace the SGD with any other optimizer 36 available in PyTorch or make your own custom one.