It depends which model you would like to use.
While some torchvision.models use an attribute for the non-linearity, such as self.act, which can be easily replaced by a new assignment, others use the functional API, so that you would need to override the forward method.