I only tried 1d case, and only tried on cpu, where Linear was 10 times faster.  Interesting that conv2d is in fact faster than linear in the 2d case. Since they are mathematically equivalent, I wonder if this means we should use pointwise conv2ds instead of Linears for certain geometries???