In the fall of 1742, Oliver De Lancey secretly married Phila Franks, the daughter of a prominent and successful New York Jewish family. For six months, they kept the match secret, but in the spring of 1743, Phila announced the union and went to live with her husband. The letters of Abigail Franks, Phila's mother, to her son Naphtali in England speak of her sense of betrayal and her pain, and she never spoke to Phila again. Her husband, on the other hand, accepted the marriage. Phila and Oliver de Lancey had at least two sons and a daughter:  He left New York for England in 1783, and died on October 27, 1785, in Beverley, Yorkshire. He was buried in Beverley Minster, where his grave and memorial can be visited.