Thanks, you are right, I forgot to set the device for input as ‘cuda’.
Now, I have another problem that I appreciate it if you guide me about it, too:
The backward function of my customized cnn is sth like this:
def backward(ctx, grad_output):
        grad_output = grad_output.detach()
        input, filter, bias = ctx.saved_tensors     
        grad_bias = torch.sum(grad_output)
        ....

and in main function is like this:
loss.backward()

I receive this error:
Traceback (most recent call last):
  File "MyCNNCustomized_v6_based_on_MyCNN13.py", line 585, in <module>

>     loss.backward(None)

File "/home/zahra/.local/lib/python2.7/site-packages/torch/tensor.py", line 107, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/zahra/.local/lib/python2.7/site-packages/torch/autograd/__init__.py", line 93, in backward
    allow_unreachable=True)  # allow_unreachable flag

> RuntimeError: expected Variable or None (got numpy.ndarray)


I do not know what the problem is. Do you have any idea?
It is interesting that the code was executable on cpu while I receive this error in gpu!
Many thanks before all