E. Cobham Brewer was the son of Elisabeth, née Kitton, and John Sherren Brewer, a Norwich schoolmaster associated with the Baptist congregation of St Mary's Chapel in Norwich. His father kept a school in Calvert Street, Norwich, until 1824, when he opened a new academy in Eaton on the outskirts of Norwich.