Solved by tom in post #2 
              
              
                How about inp.view(4, 3, 3, 3).permute(0, 2, 1, 3).reshape(1, 12, 9)? 
(If anything, unfold would be used when getting into the other direction with overlapping patches.)