 Colby was elected as a Liberal-Conservative MP in the House of Commons of Canada in 1867 representing Stanstead and remained in parliament until his defeat in 1891.He served as President of the Privy Council under Sir John A. Macdonald from 1889 to 1891 and was previously Deputy Speaker and Chairman of Committees of the Whole of the House of Commons.Colby supported the introduction of tariffs to reciprocate against those imposed by the United States.