So I knew Tensor.unfold(dim, size, step) (which is different from nn.functional.unfold) can produce the type of overlapping patches you want. Initially, I thought it would be step 2 but this will make the 0 2 4 6, so we need step 1 and then skip every other slice with the ::2. The size = 6 is because you asked for 3 outputs but as we are skipping every other we need 6 (probably 5 would do in hindsight). Then the :4 is to limit each row to 4 elements.
There also is the - perhaps more straightforward alternative - to use a.as_strided((3, 16, 4), (2, 10, 1)) to specify the layout in memory directly but I am never sure how much to trust as_strided with autograd. Quite likely, everything just works, but at some point in the distant past, it did something that I found strange. In that sense Tensor.unfold seemed more natural even if it looks like black magic with all these numbers derived through iterative fiddeling.
Best regards
Thomas