Two years later, he married Ellen Anderson, daughter of a Wellington architect. Ballance first contemplated moving into national politics in 1872, putting his name forward as a candidate for the seat of Egmont in a parliamentary by-election. However, Ballance withdrew from the ballot before the vote was held. In 1875, Ballance entered Parliament, having won Rangitikei in a by-election. He campaigned on two major issues – the abolition of the provinces (widely regarded as incompetent, petty, and obstructive) and the provision of free education. Ballance soon made his presence felt in Wellington. The abolition of the provinces occurred in 1876 under Julius Vogel—after which Ballance turned his attention to promoting closer land settlement, considering it the main political issue of the day. In 1877, Ballance entered the cabinet of Sir George Grey, a former Governor who was then Premier. Grey's policies were not closely aligned with those of Ballance, but Ballance believed that he could nevertheless accomplish something worthwhile. He was Minister of Customs, Minister of Education, and later Colonial Treasurer. His appointment to head the treasury was a surprise to most, giving a high office to a relative newcomer on the political stage. On 6 August 1878, Ballance delivered a financial statement, which was seen as the most significant since the public works announcement by Julius Vogel in 1870. Ballance set about reforming the tariff system by removing duties on basic necessities and introducing a modest though somewhat symbolic land tax, an idea he later revisited. His alliance with Grey ended with a notorious and very painful quarrel. Ballance found Grey far too controlling and authoritarian, resigning his portfolios yet still gave him confidence in the house. From 1879 Ballance represented Wanganui, but in 1881 he lost by just four votes (393 to 397), and it was reported that seven of his supporters were too late to vote as their carriage broke down. He returned to Parliament for Wanganui in 1884. On re-election as an Independent in 1884, Ballance became a minister in the Cabinet of Robert Stout, a fellow liberal. He was Minister of Lands and Immigration, Minister of Defence and Minister of Native Affairs (relations with Māori). In his role as Minister of Lands he encouraged intensive settlement of rural areas, aiming to increase the number of people leaving the cities to "work the land", which he believed was essential to increase productivity and self-sufficiency. His system of state-aided "village settlements" – small holdings were leased by the Crown to farmers and money lent them to make a beginning of building and cultivation – was generally successful.