Hello.
It will depend on how the original model is implemented.
If it creates modules for the ReLU/batchnorm during the initialization, you can just replace these modules wherever they are and then the forward method will use your new modules instead.
If you use the functional interface for ReLU directly in the forward() method of the Module and do nn.functional.relu(). Then you will have to modify the forward() function itself to replace these calls with the activation function you want to use.