Solved by tom in post #2 
              
              
                In 
self.Bi_weight = Parameter(torch.Tensor(self.weight.shape)).cuda()

the .cuda() is computation and you don’t have a Parameter in self.Bi_weight. 
Use self.Bi_weight = Parameter(torch.Tensor(self.weight.shape).cuda()) or better yet just leave the .cuda() alone and do model.cuda() at the end. 
Bes…