Weber was born into an academic background, with his father serving as a professor at the University of Wittenberg. Weber became a doctor, specializing in anatomy and physiology. Two of his younger brothers, Wilhelm and Eduard, were also influential in academia, both as scientists. One did physics and the other anatomy. Ernst became a lecturer and a professor at the University of Leipzig and stayed there until his retirement.