 In 1873, he married his distant cousin, Caroline Augusta Cabell, who died in 1876.He remarried to Caroline Cabell's sister, Sarah Randolph Cabell, in 1886.Brown studied the early history of Virginia from the standpoint of the Virginia Company and became convinced that the early history of the commonwealth had not been truly written.