Hi all,
I want to implement a regulizer that keeps the weights in a certain range, and does not push the weight itself to become zero, but push the |w(t)-w(t-1)|>a to zero. It means that in that range, I don’t want to have any effect on the training.