There is most likely a difference in computation in particular if you are using CUDA operations. E.g. convolutions would be dispatched to cudnn, if you are using an NVIDIA GPU, which could internally call into cublas (same as in the linear layer), but isn’t guaranteed.
I don’t know, which methods are exactly called on the CPU.
For my code snippet the convolution would use cudnn::cnn::implicit_convolve_dgemm, while the linear layer would call into volta_dgemm_128x64_tn.