After serving as governor, Wise settled with Mary and his younger children in 1860 at Rolleston, an 884 acres plantation which he bought from his brother John Cropper Wise, who also continued to live there.