smith was born in roxbury, connecticut. he was the nephew of nathaniel smith and nathan smith. smith completed preparatory studies and graduated from yale college in 1815. he studied law at litchfield law school and was admitted to the bar in 1818, commencing practice in litchfield, connecticut. he married maria cook on june 2, 1832, and they had three children, catherine marie smith, jeannie penniman (jane) smith, and george webster smith. his wife, marie, died on april 20, 1849. he married mary ann dickinson walker on november 7, 1850, by whom he had six children, truman houston smith, samuel hubbard smith, edmond dickinson smith, robert shufeldt smith, henry humphry smith, and allen hoyt smith.