Mr Johnston, 69, will be the Queen's representative in Canada, charged with performing the constitutional role she fulfils in Britain.
In a speech after an pomp-filled ceremony in Ottawa, the prominent legal scholar and academic administrator called Canada "the envy of the world".
Mr Johnston replaces Michaëlle Jean, a former journalist appointed in 2005 who is to be UN envoy to her native Haiti.
"I see my role as a bridge in bringing people of all backgrounds  and ages together to create a smart and caring nation, a nation that will inspire not just Canadians but the entire world," he said.
Mr Johnston was born in northern Ontario, the son of a hardware store owner. He has had a long career in the Canadian legal academy, with postings at some of the nation's top law schools.
David Johnston has been sworn in as the 28th governor general of Canada.