As far i know …when you have to preserve the spatial information like in the semantic segmentation…use convolution. And when you need not to do anything related to spatial information like in basic classification(mnist,cat dog classifier) go with Linear layers.