Could the tensor.unbind() function be used on .data, to work per sample basis?
EDIT: For everyone interested, after some more research, I am going to use batch_size = 1, use any condition that I only dream of  and later use this approach: Increasing Mini-batch Size without Increasing Memory | by David Morton | Medium 4 to achieve a larger batch for gradient backward passing