If I understand correctly, you’re going to save some intermediate values into tmp_conv right? In that case the Variable shouldn’t require grad, because you’re going to overwrite the original content anyway. But I think it would be much cleaner and simpler to use torch.cat or torch.stack.
h_init also doesn’t need to require grad, because you won’t even be optimizing it.