Solved by ndwork in post #2 
              
              
                I have found the answer to my own question (in case others would like to know).  The code can be altered as follows to get the desired behavior: 
optimizerCopy = optim.SGD( netCopy.parameters(), lr=0.1, momentum=0.0 )
optimizerCopy.zero_grad()
for paramName, paramValue, in net.named_parameters():
  …