Hi,
You seem to be using unfold a lot. It is usually only called once for a convolution operation.
I can see why calling it multiple times in a chained manner would be very memory hungry. I’m sure the size of guide_im and x are quite large no?

Moreover, applying the optimizer is very slow.

That is surprising. Do you have a small code sample (30-40 lines) that reproduces this?