Later, after the crusaders were forced to withdraw from Egypt in September 1221, Oliver remained behind, staying until September or October 1222 in Akkon. During this time he sent two letters, one to Sultan al-Kamil of Egypt, the other to the Islamic scholars there, in which he tried with the polemical erudition of his time to convince them to reject of the Muslim faith and accept Christianity in its stead.