Yes, usually parameters are initialized with a specific initialization method to random values.
E.g. the parameters of nn.Linear are reset in these lines of code 153.
If you want to use an init method, have a look at torch.nn.init.