 Brown was employed as the master gardener at Mount Gulian from 1826 to 1864.In September 1826, he returned to Maryland and purchased his wife's freedom with money he had saved while working up north.Around 1826 James Brown began to keep a detailed journal of everyday life, one of the very few journals of daily life as experienced by a black person anywhere in the North.