Solved by rasbt in post #2 
              
              
                The weight_decay parameter adds a L2 penalty to the cost which can effectively lead to to smaller model weights. It seems to work in my case: 
import torch
import numpy as np

np.random.seed(123)
np.set_printoptions(8, suppress=True)

x_numpy = np.random.random((3, 4)).astype(np.double)
w_numpy = np…