You could simply write an initialization function, which could look like this:
def reset_parameters(self):
    for m in self.modules():
            if isinstance(m, nn.Conv2d):
                nn.init.kaiming_normal_(m.weight, mode='fan_out', nonlinearity='relu')
            elif isinstance(m, nn.BatchNorm2d):
                nn.init.constant_(m.weight, 1)
                nn.init.constant_(m.bias, 0)

and then call toy.reset_parameters() at the beginning of each fold.

Note: The example has been taken from the torchvision github repo 24 and does not contain an initialization for torch.nn.Linear. You would have to adapt this for your usecase.