Armsby was born in Northbridge, Massachusetts, on September 21, 1853, the only child of cabinetmaker Lewis Armsby and Mary A. Prentiss. Armsby earned his Bachelor of Science degree from Worcester County Free Institute of Industrial Science in 1871 and taught chemistry there for a year. He studied for two years at Yale University's Sheffield Scientific School, graduating in 1874 with his Bachelor of Philosophy degree.