This is a small function I wrote that seems to work with a lot of the torchvision models. It tries to find all ReLU layers and convert them to Softplus:
def convert_relu_to_softplus(model):
    for child_name, child in model.named_children():
        if isinstance(child, nn.ReLU):
            setattr(model, child_name, nn.Softplus())
        else:
            convert_relu_to_softplus(child)