As I said please forget about loss! For each operation we have a forward function which we can call as easy as calling for ex: torch.add(x,y). And we have backward function which I dont know how to call manually for example something like torch. AddBackward(ctx,x,y)! I know that autograd call them one by one somehow but I dont know how to call it my self!