The museum occupies a two-story house that was constructed in 1891 by a local businessman, James Bouton. It was built as an investment two years after the Long Island Rail Road opened the Murray Hill railway station nearby. The house was then purchased in 1899 by a German immigrant named Conrad Voelcker, who moved in with his wife Elizabeth and infant daughter Theresa.