Is there a quicker way then listing out all the indexes you want to change?
This works:
model.features[2] = NewActivation()

But this is closer to what I want to do, however the assignment doesn’t work:
for i, (name, layer) in enumerate(model.named_modules()):
    if isinstance(layer, nn.ReLU):
        layer = NewActivation()