The estate was established by Robert Livingston following the death of his father, the first Lord of Livingston Manor, in 1728; while most of the manor was inherited by the eldest son Philip Livingston, 13000 acres in the southwest corner, later named Clermont, was willed to Robert.