Solved by jmandivarapu1 in post #2 
              
              
                The weight_decay parameter adds a L2 penalty to the cost which can effectively lead to to smaller model weights. 


https://dejanbatanjac.github.io/2019/07/02/Impact-of-WD.html