The first thing that happens in my model forward method is calling checkpoint few times using several feature extractors.
However, I get the following warning:
 UserWarning: None of the inputs have requires_grad=True. Gradients will be None
  warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")

This issue seems to be described here:

  
      github.com/pytorch/pytorch
  
  
    



Issue: checkpoint(function, *args) should have the same requires_grad as function(*args) 72


	
	opened by ruotianluo
	on 2018-05-16
	
	
	


Issue description
In principle, to my understanding, checkpoint(function, *args) should behave exactly the same during forwarding as function(*args).
However, now, if the all...




  
  
    
    
  
  


Is there any recommended workaround for this right now ?
At first I thought about doing something like:
input_data.requires_grad_(True)
and then do:
y = checkpoint(foo, input_data)
But I think that it will also treat the input as parameters (like done in sensitivity/saliency map creation), which isn’t something I need or want to add to the computation.
Should I add/use some sort of a dummy function before calling checkpoint?
any suggestions?