Hi apytorch,
You can shuffle the samples in the range of 2x batch size on the sorted samples, that’s what I mean “local random”.
I just discuss the issue about padding and masking a few days ago. You can find it here.

  
    
    
    Padding and masking in convolution autograd
  
  
    Hi, I’m using pytorch to do some encoding things on 1-D inputs by 1-D convolution. 
I have 2 questions since the length of inputs is inconsistent in a batch. 
1.Currently, I maintain some mask tensors for every layer to mask their outputs by myself. And in my code, I have to compute the change of each mask tensor in the forward method since the size of input and output may be different. 
Q:Is this a regular way? 
2.Now, I just mask the outputs for each layer without caring about gradients. 
Q:Is…