I am unsure of exactly what you are looking for in this (if you can give some example, that will definitely help). One way to achieve this would be to initialise a parameter randomly and always apply a tanh activation function to it before consuming it. This way, you will ensure both the range (-1, 1) and updates from gradients.
Here is a simple example to demonstrate this:
# initialise randomly from normal distribution
>>> myparameter = torch.randn((100, 100))
>>> print(torch.unique(myparameter))
tensor([-3.7089, -3.5629, -3.4922,  ...,  3.2828,  3.4450,  3.5880])

#  normalise using tanh activation
>>> myparameter_norm = torch.tanh(myparameter)
>>> print(torch.unique(myparameter_norm))
tensor([-0.9988, -0.9984, -0.9981,  ...,  0.9972,  0.9980,  0.9985])

# proceed to use myparameter_norm as input to next compute