It’s product of the way you define it.
gpu_w itself is not leaf as it’s a copy of the cpu tensor. The cpu version is leaf in fact.
You can bypass this by calling
self.w = nn.Parameter(torch.ones(1).cuda())
Edit :
It may be better you to define that parameter on cpu and then to allocate on gpu if you use it during forward function. That makes much more sense. In fact, you can also wrap it inside another nn.Module in order to allocate it into gpu inside init.