Solved by SantMan in post #2 
              
              
                Hi All, 
Got the answer myself. Because we are using TORCH_MODULE() wrapper.  Which creates a shared pointer for us. Hence instead of m.to(device), we should use m->to(device). Notice the -> insetd of .(dot). That builds as expected.