About scenario 1, we are ok right?
But about case 2, I don’t know why it does not accumulate gradients even it computes for it. In docs, literally says The fact that gradients need to be computed for a Tensor do not mean that the  grad  attribute will be populated, see  is_leaf  21 for more details. (source) 2
But about case 3, logically, if we want to update weight and bias, we need to corresponding gradients, but about code, I am not sure because we can trick scenario 2 to be similar to scenario 3 using defining tensors directly into GPU instead of copying from CPU, in this way, case 2 will be a leaf and will have grad. Also, weight and bias n nn.Linear are parameters a specific case of tensors, that is, maybe that is the reason that enables this case.
a = torch.rand(3,2,requires_grad=True, device='cuda:0')
b = torch.rand(2,3).cuda()
loss = 10 - (a @ b).sum()
loss.backward()
print(a.is_leaf)
a.grad

# output
True
tensor([[-1.4735, -1.9717],
        [-1.4735, -1.9717],
        [-1.4735, -1.9717]], device='cuda:0')

I really need to study a few things, sorry for my lack of knowledge!