Solved by googlebot in post #6 
              
              
                checkpoint is for cases where at least one argument has requires_grad=True, using it like that skips self.model(i.e. resnet) training.