Krish:


Thanks for reply Krish. Previously I was calculating loss in float variable. I have updated the a part of code such that now loss is a tensor. Here are the details of the loss updates;

Initially I am calculating loss which is a tensor;

tensor(1.9175e-11, dtype=torch.float64, grad_fn=<PowBackward0>)


I am averaging 100 losses in variable batch_loss,

tensor(3.7276, dtype=torch.float64, grad_fn=<AddBackward0>)


Then I am averaging 25 batch_loss in variable train_loss;

tensor(0.7455, dtype=torch.float64, grad_fn=<AddBackward0>)

When I use train_loss.backward(), it shows error.
RuntimeError: Function AddBackward0 returned an invalid gradient at index 1 - expected type TensorOptions(dtype=float, device=cuda:0, layout=Strided, requires_grad=false) but got TensorOptions(dtype=float, device=cpu, layout=Strided, requires_grad=false) (validate_outputs at /opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/autograd/engine.cpp:484)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x4e (0x7f5692314b5e in /home/mz/anaconda3/envs/semSeg/lib/python3.7/site-packages/torch/lib/libc10.so)
frame #1: <unknown function> + 0x2ae2834 (0x7f56bbfd8834 in /home/mz/anaconda3/envs/semSeg/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #2: torch::autograd::Engine::evaluate_function(std::shared_ptr<torch::autograd::GraphTask>&, torch::autograd::Node*, torch::autograd::InputBuffer&) + 0x548 (0x7f56bbfda368 in /home/mz/anaconda3/envs/semSeg/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #3: torch::autograd::Engine::thread_main(std::shared_ptr<torch::autograd::GraphTask> const&, bool) + 0x3d2 (0x7f56bbfdc2f2 in /home/mz/anaconda3/envs/semSeg/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #4: torch::autograd::Engine::thread_init(int) + 0x39 (0x7f56bbfd4969 in /home/mz/anaconda3/envs/semSeg/lib/python3.7/site-packages/torch/lib/libtorch_cpu.so)
frame #5: torch::autograd::python::PythonEngine::thread_init(int) + 0x38 (0x7f56bf31b558 in /home/mz/anaconda3/envs/semSeg/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
frame #6: <unknown function> + 0xc819d (0x7f56c1d7e19d in /home/mz/anaconda3/envs/semSeg/lib/python3.7/site-packages/torch/lib/../../../.././libstdc++.so.6)
frame #7: <unknown function> + 0x76db (0x7f56dac2e6db in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #8: clone + 0x3f (0x7f56da95788f in /lib/x86_64-linux-gnu/libc.so.6)


Then I use the Variable() as given blow;

train_loss_tensor = Variable(train_loss, requires_grad=True)
#tensor(0.7810, dtype=torch.float64, requires_grad=True)

print(train_loss_tensor.grad_fn)
#None

train_loss_tensor.backward()

It is executing (no error), but still model is not training. Will you please elaborate, how could I make it working? Thank you.