I tried to copy from the tensor to a new leaf variable, like b = torch.tensor(a.datach(), requires_grad=True), however, the REPL adviced me:
/home/acgtyrant/.local/bin/ipython:1: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).

But I think sourceTensor.datach().clone().requires_grad_(True) will be better than sourceTensor.clone().requires_grad_(True) while any function like clone or cuda will create some subgraph before call detach. So detach first, then call some functions which do not have any side effect, then call requires_grad_(True) to create a pure leaf variable finally, like sourceTensor.detach().index_select(0).clone().cuda(1).requires_grad_(True).