Gocha!
python decorator (e.g. @timeit )  decorated def forward() was causing the problem. I think anything related with context manager could mess up cuda context(?is the term right?) so that it makes the error reported above. How could this happen!
is this because that I called sth related to torch.cuda.somefunction() inside the decorator routine?