marioo:

I need to add .clone() after the first SeLU, if I added it in the next line: x[mask] = mut(x.clone(), w2, mask) it does not work. Why is this?


let’s disambiguate things first, this is working:
a = F.selu(x)
b = a.clone()
b[mask] = mut(b, w2, mask)
b[mask] = F.selu(b[mask])
b[mask] = mut(b, w3, mask)
your breaking change:
a = F.selu(x)
b = a.clone()
a[mask] = mut(b, w2, mask)
a[mask] = F.selu(a[mask])
a[mask] = mut(a, w3, mask)
This suggests that “a” is referenced in some backward function, that prevents "a[mask] = " ops, but this problematic reference doesn’t exist in the first code with “b”.
Obvious culprit is F.selu itself, that keeps a reference to its output. It is hard to tell when this is the case - autograd code generation manages stored tensors, AFAIK this is not documented.
Relevant entry in tools/autograd/derivatives.yaml:
self: elu_backward(grad, alpha, scale, input_scale, result)



 marioo:

Also, it seems that I only need to use .clone() before the first call to mut() , but not before the second time. Why?


You’re accidentally avoiding problems with mut() because x[mask] in it copies values from x to a new memory. Note that this behaviour is different from “x[mask] = …” in-place assignments, handled by __setitem__.