By calling the to() operation on nn.Parameter you could create a non-leaf parameter.
Move the device transfer into the tensor creation:
self.combine_weight = nn.Parameter(torch.ones(12).to(device) / 12, requires_grad=True)

Let me know, if that helps.