ADONAI_TZEVAOT:

So when the parameter updates will it only take values from randn?


No, since the gradient determines the updates, not the initial values.



 ADONAI_TZEVAOT:

Also since 0 is one of the values in randn , how to make sure that none of the values in the tensor is 0?


The probability to sample a “perfect” 0.0 value is not very high, but in case you want to make sure no zeros are there, you could replace them before creating the parameter:
x = torch.randn(10)
x[0] = 0.0
zero_idx = x==0.0
while zero_idx.any():
    x[zero_idx] = torch.randn_like(x[zero_idx])
    zero_idx = x==0.0

param = nn.Parameter(x)