Hi ! I was trying to add new parameters a,b in every layer of my network as follows:
self.a = nn.Parameter(torch.ones([1,1],dtype=torch.double), requires_grad=True)       ###
self.b = nn.Parameter(torch.ones([1,1],dtype=torch.double)*torch.sqrt(torch.Tensor([self.dim]).double()), requires_grad=True)       ###
self.register_parameter('aa',self.a)                                                                 ###
self.register_parameter('bb',self.b) 

I want to initialize them separately, say self.a=1, self.b = sqrt(dim). But as it turns out self.a and self.b are initialized with other parameters in my network: self.a = -1.74 self.b=0.79. Do anybody know how to
initialize self.a, self.b separately, avoid initialization with other parameters in the network?
Also, I find that self.aa, self.bb have different data types from self.a, self.b when using GPU. How to solve this?