@albanD Thanks, you solved my problem in another approach.
However, If i must write tensor as a = torch.tensor(..).to(device), like the following linear regression example:
def main2():

	device = torch.device('cuda')
	lr = 5e-2

	w_ = torch.tensor(1., requires_grad=True)
	b_ = torch.tensor(0.1, requires_grad=True)
	w = w_.to(device)
	b = b_.to(device)


	criteon = nn.MSELoss()
	optimizer = optim.Adam([w_, b_], lr=lr)

	for i in range(500):
		x = torch.rand(1).to(device)[0]


		pred = w * x + b
		y = 2 * x + 3
		loss = criteon(pred, y)

		grads = torch.autograd.grad(loss, [w_, b_])
		w_.grad.fill_(grads[0])
		b_.grad.fill_(grads[1])
		optimizer.step()

		print(w_, b_)

It print the Error:

Traceback (most recent call last):
File “/home/i/ncrs/aaai/fsgan/test2.py”, line 94, in 
main2()
File “/home/i/ncrs/aaai/fsgan/test2.py”, line 60, in main2
w_.grad.fill_(grads[0])
AttributeError: ‘NoneType’ object has no attribute ‘fill_’

What should I do? How to make the above episode code work as normal?