If it had been Inglethorp who was carrying on an intrigue with Mrs. Raikes, his silence was perfectly comprehensible. 
It was incomprehensible that Inglethorp should remain silent, if he had been having an affair with Mrs. Raikes. 