The main house burned down during a dinner party in 1766 and the remaining structure sat dormant for many years until New York lawyer Jonathan Prescott Hall built a new roughly 5800 sqft castellated residence directly on top of the old ivy-covered ruins.