The visitors moved ahead when Tom Rogic volleyed in Kieran Tierney's cross.
Inverness equalised when Billy King cut inside from the left to send an impressive shot into the far corner.
Scott Sinclair punished Gary Warren's mistake with Celtic's second and though they dominated thereafter, Alex Fisher headed Inverness back on terms.
Brendan Rodgers' side, who have a game in hand, extend their lead over Hearts and St Johnstone to three points, while Caley Thistle move a point above Partick Thistle, who slip to bottom again.
The result completed a mixed week for Celtic, who beat Rangers 5-1 last week before losing 7-0 to Champions League opponents Barcelona on Tuesday.
Media playback is not supported on this device
Celtic are simply irresistible at times on the front foot.
For the opener, Tierney broke clear down the left, measured a lovely ball with the outside off his boot and Rogic converted. Simple, ruthless, effective.
Celtic's second was executed beautifully. Sinclair seized on a defensive error, roared into open space and produced a magnificent curling finish.
Sinclair - with a goal in each of Celtic's five league games - could not have dreamed of a better start following his move from Aston Villa.
The visitors should certainly have added to their tally but goalkeeper Owain Fon Williams was inspirational in deflecting shot after shot away from his net.
He produced strong saves to deny Tierney, Moussa Dembele and Callum McGregor, whose effort was tipped on to the bar and over.
Sinclair saw another tremendous curling effort tipped away, Erik Sviatchenko nodded against the post and McGregor's chip was blocked by Fon Williams.
Tierney whacked another off the bar and Stuart Armstrong fired wide. It was punishing at times at the back for the home side.
On this form, Celtic will take some stopping but Inverness somehow managed to recover.
It would have been easy for the home side to wilt but they got back into the game in the first half thanks to a tremendous King shot.
Celtic keeper Dorus de Vries might have done better than touch the ball into the corner, though King deserves all the plaudits.
A crucial moment in the match arrived soon after. Ross Draper nicked possession off McGregor and raced clear. On the edge of the box he went to ground under Sviatchenko's challenge.
It seemed inevitable referee Don Robertson would blow his whistle for a foul but play continued.
It was a pivotal moment which went in Celtic's favour but they could not fully take advantage.
Inverness held on and equalised when substitute Fisher headed in Greg Tansey's delivery to make it three games unbeaten for Richie Foran's side.
Inverness will be content with how their back-line coped with Celtic's onslaught.
Rodgers may be less so with his own sides' defensive performance as for the fifth straight Premiership match they failed to earn a clean sheet.
Kolo Toure was rested. Sviatchenko looked less than sharp. Jozo Simunovic appeared as a second-half substitute for a long-awaited run-out following long-term injury.
On another day their front men would have had the match won but it's an area that arguably cost Celtic two points in the Highlands.
Match ends, Inverness CT 2, Celtic 2.
Second Half ends, Inverness CT 2, Celtic 2.
Attempt saved. Moussa Dembele (Celtic) header from very close range is saved in the bottom right corner.
Foul by Jake Mulraney (Inverness CT).
Mikael Lustig (Celtic) wins a free kick in the defensive half.
Attempt missed. Greg Tansey (Inverness CT) right footed shot from outside the box is close, but misses to the left from a direct free kick.
Erik Sviatchenko (Celtic) is shown the yellow card for a bad foul.
Aaron Doran (Inverness CT) wins a free kick in the attacking half.
Foul by Erik Sviatchenko (Celtic).
Goal!  Inverness CT 2, Celtic 2. Alex Fisher (Inverness CT) header from the centre of the box to the bottom left corner. Assisted by Greg Tansey with a cross.
Corner,  Inverness CT. Conceded by Dorus de Vries.
Attempt saved. Aaron Doran (Inverness CT) right footed shot from outside the box is saved in the bottom right corner.
Billy King (Inverness CT) wins a free kick on the right wing.
Foul by Kieran Tierney (Celtic).
Attempt missed. Stuart Armstrong (Celtic) right footed shot from the centre of the box misses to the left.
Substitution, Celtic. Stuart Armstrong replaces Tomas Rogic.
Kieran Tierney (Celtic) hits the bar with a right footed shot from the left side of the box.
Substitution, Inverness CT. Alex Fisher replaces Lonsana Doumbouya.
Attempt missed. Callum McGregor (Celtic) left footed shot from outside the box is too high.
Tomas Rogic (Celtic) wins a free kick in the defensive half.
Foul by Gary Warren (Inverness CT).
Attempt missed. James Forrest (Celtic) right footed shot from the right side of the box misses to the left.
Corner,  Celtic. Conceded by Owain Fon Williams.
Attempt saved. Callum McGregor (Celtic) right footed shot from the centre of the box is saved in the top right corner.
Aaron Doran (Inverness CT) wins a free kick on the right wing.
Foul by Callum McGregor (Celtic).
Substitution, Celtic. James Forrest replaces Patrick Roberts.
Aaron Doran (Inverness CT) wins a free kick in the attacking half.
Foul by Scott Brown (Celtic).
Erik Sviatchenko (Celtic) hits the left post with a header from the centre of the box following a corner.
Corner,  Celtic. Conceded by Carl Tremarco.
Corner,  Celtic. Conceded by Jake Mulraney.
Foul by Greg Tansey (Inverness CT).
Callum McGregor (Celtic) wins a free kick on the right wing.
Substitution, Inverness CT. Jake Mulraney replaces Liam Polworth.
Substitution, Celtic. Jozo Simunovic replaces Cristian Gamboa.
Corner,  Celtic. Conceded by Owain Fon Williams.
Attempt saved. Scott Sinclair (Celtic) right footed shot from outside the box is saved in the top right corner.
Corner,  Celtic. Conceded by Owain Fon Williams.
Attempt saved. Tomas Rogic (Celtic) left footed shot from outside the box is saved in the bottom left corner.
Celtic dropped points in the Scottish Premiership for the first time this season as Inverness Caledonian Thistle netted a late leveller.