From the below URL’s I was able to piece together that if im using a manual K-fold (Not skorch) that I need to reinitialize the weights before each new fold, this is the suggestion on how to do that,

  
      stackoverflow.com
  
  
      
    
  

  How to initialize weights in PyTorch? 10



  python, neural-network, pytorch



  asked by
  
  
    F&#225;bio Perez
  
  on 04:34PM - 22 Mar 18


  
  
    
    
  
  



  
    
    
    What is the best way to apply k-fold cross validation in CNN? 
  
  
    Hello, 
How can I apply k-fold cross validation with CNN. I do not want to make it manually; for example, in leave one out, I might remove one item from the training set and train the network then apply testing with the removed item. Could you please help me to make this in a standard way. 
Regards,
  


I must be missing something though,
class Net(nn.Module):

    def init_weights(self):
        if (type(self) == nn.Linear):
            nn.init.xavier_uniform(self.weight)
            self.bias.data.fill_(0.01)

    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(4, hl)
        self.fc2 = nn.Linear(hl, 3)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return x

net = Net()
net.apply(init_weights) # reset network weights

NameError: name ‘init_weights’ is not defined