If you have a working model, e.g. sklearn.linear_model.Ridge 23 make sure to dig a bit into the model and then you could try to reimplement it in PyTorch.
A lot of sklearn models use some regularization, which proved to work good, while these techniques are often forgotten in the custom PyTorch implementation.