Fowler was born in Atlanta, Georgia. He attended Davidson College, and then entered the U.S. Army as an intelligence officer. After discharge, he returned to school to earn a law degree from Emory University School of Law.