I’m not sure, how the example result would fit into a [4, 1] window, but you could try to use tensor.unfold, which will not collapse the patches into a dimension:
tst_tensor = torch.tensor([[1, 1, 1, 2, 3, 1],
                        [1, 2, 0, 0, 0, 0],
                        [1, 0, 1, 0, 0, 0],
                        [5, 5, 5, 5, 5, 5],
                        [5, 5, 5, 0, 0, 0],
                        [0, 2, 3, 0, 0, 0]])

patches = tst_tensor.unfold(1, 4, 2)