Thanks for the answer. Do you know if this handmade conv will give exactly same results with Conv2d? When we train it for some task, will they be exactly same? Or are there any other hidden optimizations/different way of calculating gradients in the cuda implementation?