@shaanchandra From the discussion above, it seems that there are two potential solutions:

As @knilox mentioned above, remove the gradient regularization loss entirely - this may seem that we are not faithfully reproducing the original CTGAN, however DP eventually clips the gradients to bound SGD sensitivity, automatically introducing some regularization. Further, it appears that the smartnoise-sdk implementation which is being followed here already does this.
Keep the gradient regularization loss but do not compute per-sample gradients during the call to autograd.grad (basically keep the backward hook disabled during that time) as it is for estimation of gradients and not for updating the network. We’ve tested this out and it works initially, however it seems to be creating a new issue during the .step() where it complains that the norms are of unequal length.
So our recommendation is do (1). Option (2)'s error is probably unrelated and is also a fix.