Your code shouldn’t work at all, since you are wrapping the forward pass into a no_grad guard:
G = nn.Linear(1, 1)
z_initial = torch.randn(1, 1)

with torch.no_grad():
   gan_out = G(z_initial)

z_initial.retain_grad = True
loss = nn.MSELoss()(gan_out, torch.rand_like(gan_out))
loss.backward()
# > RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

Could you explain the use case a bit and why you are disabling Autograd while trying to calculate the gradients?