But you do need these inputs to compute the gradient wrt to the weights (which are the Tensors that you are actually learning).
Also in the model above, the conv inputs are the output of relu. And the relu also needs this output to compute its gradient. So that Tensor is saved once for both ops: the conv wrt weights and the relu.