 He agreed, however, to return to cabinet as a minister without portfolio under Mackenzie Bowell and then Sir Charles Tupper until the Conservatives were defeated in the 1896 federal election.In politics, Smith was an advocate of the rights of Irish Catholics and lobbied for the appointment of Irish Catholics to patronage positions.In 1872, he lobbied Macdonald for the gradual release of Fenian raiders who had attempted military attacks on Canadian soil in order to promote Irish independence.