If the activation function (or generally, the module) does not contain any parameter of buffers, you could just reuse them.
Some activation functions, such as PReLU contain trainable parameters and should thus not be reused (only if this explicitly fits your use case).