It seems that there is no permutations that can make it work.
I have tried with the following snippet to generate all permutations and check if there is one that is equal to tf.space_to_depth(t,2).eval()

for p in list(itertools.permutations([0,1,2,3]))
        t.permute(p[0], p[1], p[2], p[3]).view(1,2,2,8)