In 1817 Storm bought the 150 acres opposite the house for his farming operations from Jacob R. Van Rensselaer, a descendant of the area's original Dutch manorial family who still had considerable landholdings in the area and was active in local politics as a state assemblyman, serving as speaker of that body and later New York's Secretary of State.