sign is a discrete function whose gradient is zero everywhere. Theoretically the workaround cannot be working.
I think you need to implement a custom backward function that would calculate the gradient of the signed weights and then copy that gradient over to the full precision weights.