Local lore dates the building's center block to 1765, when Benjamin Whitaker owned what was then a 129 acres property but the hand-hewn timbers visible in the basement, and the finishes used are not consistent with that period.