A better way to accomplish this might be to use hooks:
import torch
import torch.nn as nn

def grads(mod):
    return list(x.grad for x in mod.parameters())

torch.manual_seed(0)

x = torch.tensor([1., 2.])
mod = nn.Linear(2, 2)
mod2 = nn.Linear(2, 2)

y = mod2(mod(x))
y.sum().backward()

print(grads(mod))
print(grads(mod2))

print("--------------")

torch.manual_seed(0)

x = torch.tensor([1., 2.])
mod = nn.Linear(2, 2)
mod2 = nn.Linear(2, 2)

for p in mod2.parameters():
    p.register_hook(lambda x: x/10)

y = mod2(mod(x))
y.sum().backward()

print(grads(mod))
print(grads(mod2))

Output:
[tensor([[-0.0768, -0.1535],
        [ 0.7478,  1.4955]]), tensor([-0.0768,  0.7478])]
[tensor([[ 0.4810, -1.4331],
        [ 0.4810, -1.4331]]), tensor([1., 1.])]
--------------
[tensor([[-0.0768, -0.1535],
        [ 0.7478,  1.4955]]), tensor([-0.0768,  0.7478])]
[tensor([[ 0.0481, -0.1433],
        [ 0.0481, -0.1433]]), tensor([0.1000, 0.1000])]


In train mode (with a model with dropouts, BN and e.t.c) is it  true that if y = model(x), then model((x,x))=(y,y') and y=y' with all  other identity matters (like in-variance to dropouts, backward path and  e.t.c)

I guess it would be the same, but a better way of doing that would just to be to clone the output y and then detach?