i was uneasy at this , for the duke had questioned both myself and some colleagues previously , in a manner that suggested to me that he suspected the existence of a secret movement to restore the stuart throne .