Thank you, could you elaborate more? I don’t understand how that would be done.
I have a potential implementation in numpy (but I guess different to what you suggested):
>>> def reverse_stripe(a):
...     a = np.asanyarray(a)
...     i, j = a.shape
...     assert i >= j
...     k, l = a.strides
...     return np.lib.stride_tricks.as_strided(a[j:], (i-j, j), (k, l-k))
... 
>>> a = np.arange(24).reshape(6, 4)
>>> reverse_stripe(a)
array([[12,  9,  6,  3],
       [16, 13, 10,  7],
       [20, 17, 14, 11]])

If I take that directly to pytorch with:
            def batch_stripe(a):

                b, i, j = a.size()
                assert i > j
                b_s, k, l = a.stride()

                # left top to right bottom
                # return torch.as_strided(a, (b, i - j, j), (b_s, k, k + 1))

                # left bottom to right top
                return torch.as_strided(a[j:], (b, i-j, j), (b_s, k, l-k))

I am again getting the error from before:
 File "J:\Anaconda_Python3_6\envs\cuda2\lib\site-packages\torch\autograd\__init__.py", line 89, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Tensor: invalid storage offset at c:\programdata\miniconda3\conda-bld\pytorch_1524546371102\work\aten\src\thc\generic/THCTensor.c:759

I guess some parameter is not tracked properly again, I guess because I am modifying a?