he could n't quite analyse why he was so cross , but perhaps it had something to do with the fact that the man in grey , far from being the mafioso or agent of a foreign power conjured up by his imagination , was just a civil servant who happened to work in her majestys revenue and customs .