Had a similar problem. For me, mark_dirty worked perfectly. Makes backward getting called without needing to clone the input tensor or store pairs of tensors with mark_shared_storage.
class ManipulateGradient(Function):
	def forward(self, input):
		self.mark_dirty(input)
		return input

	def backward(self, grad_out):
		# manipulate gradient here
		return grad_out + 0.42