Popularly known as Jokowi, the 53-year-old took the oath of office at a ceremony held at parliament in Jakarta.
He was then cheered through the streets as he made his way on a horse-drawn carriage to the state palace.
The former Jakarta governor is the first president not to have come from the military or political elite.
Elected in July, the former furniture exporter and son of a carpenter now leads the world's third-largest democracy, with a population of about 250 million people.
US Secretary of State John Kerry and Australia's Prime Minister Tony Abbott were among those who attended the inauguration.
"We used to have our transfers of power marked by bloodshed," said an elderly Indonesian man to me today. "Today - we have this: a huge party."
Indonesia is celebrating the inauguration of its new president in style, with dancers decked out in traditional costumes, a marching band and thousands of people lining up to catch a glimpse of Jokowi on the main thoroughfare.
One man told me he had taken the day off work and brought his young daughters to witness this event because they should know what kind of man makes a "good president". Another woman, a school teacher, said that Jokowi was someone "like her" and that's why she is so thrilled he's become the leader of her country.
It was smiles all around, and a real feeling of festivity in the air. Although Mr Widodo has tough challenges to face in the future, today it was about celebrating the moment and leaving the hard work till tomorrow.
After reading the oath of office, he told Indonesians that "unity and working hand in hand are prerequisites for us to be a great nation. We will never become a great nation if we are stuck with division".
"This is a historic moment for us all to move together, to work and work,"  he said.
Mr Widodo then travelled through the capital in a horse-drawn carriage to the state palace with Vice-President Jusuf Kalla.
He was met by former president Susilo Bambang Yudhoyono at the palace. The two had met on Sunday for a tour of the estate.
An outdoor concert featuring rock bands is scheduled for Monday night, with Mr Widodo expected to appear on stage.
About 24,000 police and military personnel have been deployed, but correspondents say that the mood in the capital is upbeat.
What does Jokowi win mean for Indonesia?
Jokowi - who has promised to focus on healthcare and education - has been catapulted to power by his "man of the people" image, our correspondent says.
But that will not be easy with an antagonistic parliament in power and he will be sorely tested in this first term in office, she adds.
Mr Widodo's defeated rival in the presidential election, Prabowo Subianto, gave the new leader his conditional support last week, in a move seen by observers as a positive sign.
But two-thirds of parliament are from Mr Subianto's coalition. He said on Friday that he would ask his party to support Mr Widodo, but also said he would not hesitate to criticise if he disagreed with him.
Mr Subianto had challenged the election results, claiming there was "massive" electoral fraud, but his case was rejected by Indonesia's constitutional court.
One of the first, and biggest, challenges Mr Widodo faces is Indonesia's at least $20bn (Â£12bn) fuel subsidy bill. He has said he plans to reduce subsidies, but the move has been met with opposition.
Joko Widodo, the charismatic outsider who won Indonesia's presidency, made a call for national reconciliation and unity as he was sworn in.