@ptrblck I somehow could NOT replicate the bug, even in my original code today. It was definitely very strange because I definitely tried split(a, 40, dim=1) as suggested before and tried all variations and could not get it to work and had to settle with torch.narrow
but as of today I changed the code to split(a, 40, dim=1) and split(a, 2, dim=1), both behaved as expected and no bug. I am not sure what went wrong yesterday but thanks for the helps. I’ll keep an eye out should it happen again.
edit: I was doing a twitch stream yesterday so I could retrieve some information about what trasnpired, but only in screenshots
            
              
                
                  
                    [Album] Imgur
                  
                
                
               51
            

if you want to look at the stream record the bug happens about 2:08:00 to 2:12:00 here: