You could profile the different steps in this module and check where the bottleneck is.
Generally, I would try to see, if the loops (using range(length) and range(x.shape[1])) can be avoided, in case these are indeed the bottleneck. Also, you might want to check unfold, which could be faster than your slicing.