Mauger, a former New Zealand back, will leave the role following Saturday's Premiership game against Northampton.
O'Connor, who won three Premierships as a Tigers coach, will take over in early April, subject to being issued a visa.
Mauger had been in charge since the long-serving Richard Cockerill was sacked as director of rugby in January.
O'Connor, 46, worked under Cockerill between 2010 and 2013, before leaving to take the top job at Leinster and subsequently going to work for Queensland Reds and Tonga.
The switch has been announced just a day after the Premiership's fifth-placed side edged out Exeter in the Anglo-Welsh Cup final to claim the club's first silverware in four years.
"We're delighted that Matt O'Connor has accepted an offer to return to Welford Road," said Tigers chairman Peter Tom. "The club has conducted an exhaustive search to find the right person.
"One of the most significant factors in our deliberations was experience of the Premiership and what it takes to win the competition.
"Matt has that experience as head coach during a successful period. The team also topped the try-scoring figures in the Premiership four times in that five-year period with Matt in charge of the attack."
Leicester wing Telusa Veainu expressed his surprise, tweeting: "With 5 games to go, really?? We don't know how good we've got it! #biggerpicture #gutted."
Analysis - 'A big hit returns'
Chris Jones, BBC Radio 5 live's rugby reporter
"The management situation at Welford Road over the past few seasons has been a confusing one.
"O'Connor was a big hit as head coach at the club, winning two Premiership titles working under Richard Cockerill, before leaving to became the main man at Leinster in 2013.
"Then Aaron Mauger came in to much fanfare, before a clash of philosophies with Cockerill led to the latter's sacking.
"Now O'Connor has returned to take full charge of team affairs, with Mauger moving on.
"O'Connor will head up the coaching, with the club changing the structure of the organisation in the absence of a director of rugby figure."
Former Australia Test centre O'Connor worked as assistant coach with Brumbies and Australia 'A' before first arriving in Leicester in the summer of 2008
In his first stint at the club, working with Cockerill, the Tigers won the Premiership in 2009, 2010 and 2013, the Anglo-Welsh Cup in 2012 and reached the 2009 European Cup Final.
Since leaving Welford Road in 2013, O'Connor has worked with Leinster, who he led to the Pro12 title in his first season, Queensland Reds and Tonga.
Former All Blacks centre Mauger had been back at Welford Road as head coach since the start of 2015-16 season.
"While preparing to welcome Matt, we are also disappointed to lose Aaron," added Tom. "Matt's strengths are in his work with players on the training pitch and we are making changes to our organisational structure to enable him to focus solely on the success of the team.
"Matt has a good record in the Premiership and had a lot of success in his previous spell here," said club captain Tom Youngs. "At the same time, it is disappointing that Mage is leaving the club. He's a good man and a good rugby man."
Former Australia international Matt O'Connor is to return to Leicester Tigers in the role of head coach, replacing Aaron Mauger.