after dinner , when the ladies had removed to the drawing room , the young fitzwilliam exhibited an impatient agitation that could only be attributed to his anguish at being separated from the object of his desire .