In 1792, Morgan Lewis, the third Governor of New York, purchased an estate covering of about 334 acres and commissioned the construction of a colonial-style house on the site of the present day mansion.