Hi albanD. Thanks for your reply. Here I use \theta to represent the parameters of the network f. So say if f=nn.Linear(a,b), then we can write f(x)=Wx, and here \theta will be W. It doesn’t make sense to pass W as an input to gradcheck(), no? Because the input to gradcheck() should be the input to the forward() function of f. Are you suggesting that gradcheck() calculates df/dx rather than df/d\theta?