Ok, one way to do this is to run two convolutions with stride e.g. (2, 1), adjusting padding accordingly, and combining the resulting tensor, cutting the memory requirement rougly in half.
Interestingly, with stride (2,2), the heuristic reverts back to memory-cheap algorithm, so you can run 4 convs and combine their output.
In any case, this is really dumb and there should be a way in torch to do this painlessly.