Thats strange, it shouldn’t shuffle along the feature dimension like that. Can you post some code?
Edit : Follow up question, along what dimension is the dataloader shuffling the values? Batch or feature?