At the start of each epoch, some of the filters were selected to be penalized with L2, if the filter selection procedure is in the training loop (loop on batch) everything is OK, but I want at the start of each epoch to select filters for regularization and then begin training , in this case I have popular error :
“RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of …”, why THIS happens any help appreciated, I know computation graph generated dynamically. my code
 crt=nn.CrossEntropyLoss()
 for epochIdx in range(self.NumOfEpoch):
            reg_params = Variable( torch.FloatTensor(0), requires_grad=True)
            for n,x in self.model.named_parameters():
                if some condition meet:
                   reg_params = torch.cat([reg_params,x[[1,2],:,:,:].view(-1)]) #index 1 and 2 is sample

            #if all filter selected with this line and above for loop commented IT WORK !!!
            #reg_params = torch.cat([x.view(-1) for x in self.model.parameters()])

            for batchIdx, data in enumerate(self.trainloader, 0):
                inputs, labels = data
                inputs, labels = inputs.to(self.device), labels.to(self.device)

                self.model.zero_grad()
                features,output = self.model(inputs)
                l2_regularization = Reg_lambda * torch.norm(reg_params, 2)
                loss=crt(output,labels) + l2_regularization
                loss.backward()
                optimizer.step()
                #....