He described corruption as a "dangerous problem", adding: "We will soon organise a conference in Kabul to organise new and effective ways to combat this problem."
He also called for the strength of Afghan security forces to be bolstered and the role of international forces reduced.
"We hope that the Afghan forces will lead the task of security and stability throughout the country in the coming five years," he said.
Inviting his main defeated rival, Abdullah Abdullah, to work with him, he called for a national gathering to help bring peace.
Speaking to the BBC in Kabul after the ceremony, Mrs Clinton said Mr Karzai's speech had set out a "very positive, comprehensive path forward".
Martin Patience BBC News, Kabul
Karzai's speech contained all the key phrases: corruption, national reconciliation and unity, and security.
The international community has demanded the Afghan leader do more to tackle rampant corruption. Mr Karzai said he would host a conference on the issue in the coming months. He promised to take a tougher line and sack officials found to be corrupt. There will also be close scrutiny of the individuals he appoints to his cabinet.
But perhaps most significant was his call for Afghan security forces to take over control in five years. The US and other allies want Afghans to take the lead (as do the Afghans). While the Afghan army is improving, some experts say it will need continued support for decades to come in areas where the insurgency is at its strongest.
But the intention is there - and that will be important in Washington and European capitals. The allies need to tell increasingly sceptical publics this is not a war without end.
"It was clear today in the speech that the president has a vision of where he wants to lead the country, and it was reassuring to people, it was exciting because it was such a statement of resolve," she said.
"But the proof is in the pudding. Now we've got to get to work and make it happen - he knows that, his ministers know that."
UK Foreign Secretary David Miliband echoed her comments saying: "Now it's deeds that matter - not just words."
The Taliban, however, condemned the inauguration, saying Mr Karzai had become president "through fraud and lies".
And Mr Abdullah - Mr Karzai's closest rival in the discredited August poll - said he was not optimistic that Mr Karzai could "meet the expectations of the Afghan people and the friends of Afghanistan".
Mr Karzai was declared Afghan president after a second round election run-off was abandoned when his sole remaining challenger pulled out, saying the vote could not be free and fair.
Widespread fraud in the 20 August first round led to Mr Karzai being stripped of the outright win he appeared to have secured.
The start of his second term coincides with mounting concern in countries that have sent troops over casualty numbers and the progress of the war itself.
The Obama administration is currently debating sending more troops, with a decision expected in the coming days.
Are you in Afghanistan? How are you feeling on the day of President Karzai's inauguration? Send us your comments.
The BBC may edit your comments and not all emails will be published. Your comments may be published on any BBC media worldwide.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.