This is also puzzling me, as running on CPU it takes up to 70% of my ram to perform it, so at least 30GB, while the GPU has absolutely no issue with this computation.

The convolution is implemented differently for CPU, which is the main reason. Convolution on CPU corresponds is what you usually find in textbooks for discrete convolution / cross-correlation. On the GPU it’s using CUDA/CuDNN, where it’s implemented as a clever estimation method in terms of using a fast Winograd fourier transforms (or even something newer/more modified) for computing convolutions