sure, I’ll explain.
When you use requires_grad=True on the input data, you are telling pytorch to calculate the gradient for it.
This is actually a practical feature in few cool use cases, but it does create an overhead if this is not what you really desire, since you usually have no intention of changing, for example, the pixels values of the input image.
The overhead is pretty straightforward, the gradient tensor is calculated for the input, which adds to both the calculation time and also the GPU memory that is used when storing this gradient.
In some cases, like small images, I guess that it won’t matter that much. However, in my use case, which involve having as input very big 3d volumes of data, it does matter a lot, because , even if we ignore the calculation time overhead, it would also add over 1 GB of gpu memory usage.