I just initialized a stateless module without assigning the input in its __init__.
Basically the module now just calls your activation function without storing any parameters.
Yes, it will be used twice. You would need to manipulate the BasicBlock implementation here 18.