I’m not quite sure if this is the ideal forum for this question, but it seems the most likely place to get an answer.
I have been reimplementing the WGAN network for use in my own problem. I’ve gone through the paper and the GitHub code, and everything seems to match (in terms of operation order). However, when I clamp the parameters to the [-0.01, -0.01] cube, I end up with discriminator losses of -0.01 for both real and fake data, and total discriminator error in on the order of 10^-6 (i.e. precision differences).
When I increase the parameter clamping to [-0.1, 0.1], I get more varied losses (they’re not completely clamped), but they still hover around 0.1.
Basically I am finding that clamping the parameters of the network restricts my loss to that clamping interval as well. Hence, I can’t seem to understand the loss graphs from the paper where the discriminator loss starts around 1.5.
Has anyone else come across this phenomenon? The only part of WGAN that I’m not using is the custom weight initialization. Does that play a key role?