I am trying to organize CycleGAN code in my own fashion and wile running (disclaimer: this is just a snipped but I hope it is self-explanatory)
def optimize_parameters(self):
    real_k = Variable(torch.rand([1, 128, 128])).cuda()
    real_w = Variable(torch.rand([1, 128, 128])).cuda()

    fake_k = self.generator_K(real_w)
    fake_w = self.generator_W(real_k)
    
    # G
    set_requires_grad([self.discriminator_K, self.discriminator_W], False)
    self.optimizer_generator.zero_grad()

    gen_K_loss = self.gan_loss(self.discriminator_K(fake_k), True)
    gen_W_loss = self.gan_loss(self.discriminator_W(fake_w), True)
    gen_loss = gen_K_loss + gen_W_loss
    gen_loss.backward()

    self.optimizer_generator.step()
    set_requires_grad([self.discriminator_K, self.discriminator_W], True)

    # D
    self.optimizer_discriminator.zero_grad()

    dis_K_loss = self.gan_loss(self.discriminator_K(real_k), True)

    dis_K_loss.backward() # <<< RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.
    self.optimizer_discriminator.step()

I get RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time. and I can not understand why is this happening. To my understanding, discriminator and generator share just the input real_k and no discriminator graph should be created whilst optimizing the generator. However, this somehow happens.
Could someone help me understand this? Thanks!