I am printing gradients of a layer of Generator, with and without using .detach(). In my thinking the gradients of weights should not change when calling discriminator_loss.backward while using .detach()(since .detach() ensures the gradients are not being backpropagated to the generator), but I am observing opposite behavior. Irrespective of usage of .detach() the before and after gradients value are different when discriminator_loss.backbard() is called. Can anyone point out where I am wrong?
        netD.zero_grad()
        #netG.zero_grad()
        # Format batch
        real_cpu = data[0].to(device)
        b_size = real_cpu.size(0)
        label = torch.full((b_size,), real_label, dtype=torch.float, device=device)

        # Forward pass real batch through D
        output = netD(real_cpu).view(-1)

        # Calculate loss on all-real batch
        errD_real = criterion(output, label)
        # Calculate gradients for D in backward pass
        errD_real.backward()
        D_x = output.mean().item()

        ## Train with all-fake batch
        # Generate batch of latent vectors
        noise = torch.randn(b_size, nz, 1, 1, device=device)
        # Generate fake image batch with G
        print(netG.main[0].weight[0][0][1:3], "generator before netG(noise)")
        fake = netG(noise)
        label.fill_(fake_label)
        # Classify all fake batch with D
        output = netD(fake.detach()).view(-1)
        #output = netD(fake).view(-1)
        # Calculate D's loss on the all-fake batch
        errD_fake = criterion(output, label)
        # Calculate the gradients for this batch
        
        # In my thinking both the following print functions should provide same values when using .detach()
        print(netG.main[0].weight[0][0][1:3], "generator before errD_fake.backward() step")
        errD_fake.backward()
        print(netG.main[0].weight.grad[0][0][1:3], "generator grad after errD_fake.backward() step")