fupanbo:

should i set them as the Variable


No, Variables are deprecated since PyTorch 0.4.0, so you can just use tensors in newer versions.



 fupanbo:

when the back propagation is running , it looks like the grad update will be effected?


torch.split and torch.cat won’t detach the tensors from the computation graph and the backward call will still work.



 fupanbo:

so the code for the part of "2X24 turn to 2X8 " should put in my network 's def forward
or in my every train epoch


It doesn’t matter where these operations are called. As long as these ops are called somewhere, the results will be the same.
I would personally put these operations into the model and just return the output, which can be passed to a criterion, but your coding style might differ.