Don't Regularize Me, Bro autograd
  
  
    Thanks for the tips, guys. Your suggestions, and looking at the .step() method of an optimizer, provide some key insight. Here’s a compact example for reference purposes, in case anyone else has this need: 
model = nn.Sequential(
  nn.Conv2d(...),
  nn.PReLU(...),
  nn.Conv2d(...),
  nn.PReLU(...),
  nn.Conv2d(...),
)

params = [
  { 'params' : model[0].parameters() },
  { 'params' : model[2].parameters() },
  { 'params' : model[4].parameters() },

  { 'params' : model[1].parameters(), 'weight_d…