Hi all,
I am totally new to pytorch and I want to know its capabilities in developing new optimization ideas.
More precisely, I have previously written a paper entitled “a new framework to train autoencoders using nonsmooth regularization” where the codes where implemented in Matlab  from scratch. Now I want to use the capabilities of pytorch in developing my idea in the above paper.
Especially I want to know the following items:
1- Can I define a dynamic cost function in pytorch? In other words consider I have a term in cost function like ||w-v||_F^2 where w is the weight matrix for a special layer and v is a fix matrix which is changed in each epoch based on a specific rule.
2- Can we evaluate the gradient of dynamic cost function with respect to network parameters?
3- Can we change the point where the gradient is evaluated? In other word consider I want to evaluate the gradient in a point where the value of weight matrix w is w+a*v where a is a constant scalar and v is a constant matrix.
Thanks.