Having worked in a university mathematics department I noticed that the pure mathematicians tended to keep themselves to themselves, for instance not attending talks by visiting mathematicians unless it was of direct relevance to their research.  In contrast, we theoretical physicists would listen to any visiting speaker just for fun - even on occasion experimental particle physicists, applied mathematicians or computing specialists.
Perhaps pure maths is just more deeply specialised and leaves no possibility of such dilettanteism.