 His brother William F. Aldrich also represented Alabama in Congress, serving three partial terms during 1896–1901 from Alabama's 4th congressional district.Aldrich was born in Palmyra and suffered from poor health as a young boy.He attended public schools and a military academy at West Chester, Pennsylvania before enrolling at the Rensselaer Polytechnic Institute in Troy, New York.