 Born in Savannah, Georgia on October 1, 1835, Anderson was the son of John Wayne Anderson, a local businessman.His siblings include Major George Wayne Anderson, Captain J. W. Anderson II, and Colonel C. W. Anderson.He was educated in the local schools before receiving an appointment to the United States Military Academy.