 Alexander was born in Sydney on 6 January 1859, to a Jewish family. He was the third son of Samuel Alexander, a British emigrant and saddler. Alexander’s father died a few weeks after his birth, and five years later the remaining family moved to Melbourne. Alexander was home-educated by tutors before entering Wesley College in 1871. Alexander entered the University of Melbourne in 1875, and subsequently won awards in arts, sciences, languages and natural philosophy. In 1877, Alexander sailed for England, in an attempt to win a scholarship at Oxbridge. He successfully obtained a scholarship at Balliol College, Oxford, and went on to win a First in 1881. The following year, Alexander was awarded a Fellowship at Lincoln College, Oxford.