Thanks for the comments!
If I create my variables by the following code, can I assume this is a clean tensor in the current version?
filters = Variable(torch.randn(8,4,3,3).cuda(), requires_grad=False, volatile=True)
x = Variable(torch.randn(2,4,5,5).cuda(), requires_grad=False, volatile=True)

I would like to loops the following computation without creating an increasingly large subgraph:
for j in xrange(100000):
    y = torch.nn.functional.conv2d(x, filters, padding=1)
    x = torch.nn.functional.conv_transpose2d(y, filters, padding = 1)

I checked the GPU memory and it seems not increasing while the code is running and I guess it’s doing what I want. But I just want to double check since in my code a lot of this kind operations will be used. Also, in an inference based model, there usually is not a ‘forward’ direction since these model follows an analysis-by-inference principle. I hope the ‘forward’ in the previous reply doesn’t mean that the optimization is unrolled into many layers, I guess it won’t in the above setting. But can anyone confirm this?
In Tensorflow this can be done by using Parameter to build one layer graph, then use python control flow to loop the computation though implementing a fully inference based model is still quite hard in Tensorflow. But since PyTorch build the graph dynamically, I’m a little confused on how to achieve this safely.
Comment: Putting Variable under autograd package is apparently a choice with the pre-assumption that PyTorch will focus on error BP based models. This assumption makes PyTorch a little narrow minded. If PyTorch can provide clean convolutions for Tensors, all the people working on inference based vision models (RBM or general graphical models, adaptive deconvNet, sparse coding etc.) will probably benefit from it. Further it will make PyTorch much more general looking (as also a mathematical library) and yet this step is so easy to make.