Solved by InnovArul in post #3 
              
              
                Just incase if you want to dig deeper on col2im: 
Native C++: pytorch/Col2Im.cpp at master · pytorch/pytorch · GitHub 
Cuda code: 
pytorch/Col2Im.cu at master · pytorch/pytorch · GitHub, 
pytorch/im2col.cuh at master · pytorch/pytorch · GitHub