Solved by Eta_C in post #2 
              
              
                x = torch.nn.functional.pad(x.view(T, N, C // M, M), (1, 0, 0, 0, 0, 0, 0, 0), value=24).view(T, N, -1)

returns the same result faster.