Hi @albanD,
Thank you for your feedback.
First, I Tried the detach suggestion. Unfortunately it did not work.
Error:

Traceback (most recent call last):
File “/home/rfelixmg/Dropbox/PROJETOS/temporary/examplar_cnn.py”, line 412, in 
model, pkg = main(pkg)
File “/home/rfelixmg/Dropbox/PROJETOS/temporary/examplar_cnn.py”, line 356, in main
bin_target=bin_target)
File “/home/rfelixmg/Dropbox/PROJETOS/temporary/examplar_cnn.py”, line 86, in train
loss.backward(retain_variables=True)
File “/home/rfelixmg/anaconda2/lib/python2.7/site-packages/torch/autograd/variable.py”, line 146, in backward
self._execution_engine.run_backward((self,), (gradient,), retain_variables)
RuntimeError: there are no graph nodes that require computing gradients

However, my next try seems to me that work well:
dist_ = dist_[(dist_ > 0.0).detach()]
dist_ = Variable(dist_.data, required_grad=True)
if batch == 0:
             if batch == 0:
                loss.backward(retain_variables=True)
            else:
                loss.backward()

The code is running fine now.
My question is if the gradient will be computed right doing Variable (dist_.data, requires_grad=True);
The main reason why I am getting only the dist_ > 0. is due to dist_ be a similarity matrix with diag(dist_) = 0
If there is any other way to get only the triu elements as a vector it would be perfect.
Cheers,
Rafa