Just debug a bit using gradcheck(), what’s finally passed to the forward() is tensor, not Variable. And in some other docs it says “It can take and return an arbitrary number of tensors.” (http://pytorch.org/docs/master/autograd.html 7)  I guess what is happening underlying is inside Function core they are all tensors, but outside of Function PyTorch convert input Variables to tensors and wraps them back into output Variables, in order to build computation graph. Somehow it is a bit confusing to new users, and I am still not sure if I am right about it.