A Tudor house on the site was demolished and rebuilt by the last Gresham baronet in the 18th century, then in 1826 was given new fronts designed by William Atkinson. A tower by Philip Charles Hardwick was added in 1856.[3]