This may be all one needs know about O'Leary: 
He grew up in Montreal, the most bilingual city in Canada, where the majority of anglophones and francophones today both speak both official languages - except for the very oldest generations still more likely to be unilingual.  So, what's O'Leary's excuse for being a Montreal native and not learning French, huh? Does he have contempt for French language and culture, maybe? One might guess so. 

O'Leary's obviously a bright enough guy he'd have been able to learn French had he chosen to.
But Montrealer O'Leary chose not to bother learning French,  and has far less excuse than say a Vancouverite for not learning French as Vancouver has so few francophones, unlike  majority-francophone Montreal.
Someone like O'Leary who appears to hold Quebecois culture & language in contempt is unfit to be Tory Party leader or P.M. of Canada