Gabriel Agbonlahor scored in the final moments of extra time at the Walkers Stadium to send Aston Villa into the fourth round of the Carling Cup.
Aston Villa's Liam Ridgewell slides in on Elvis Hammond
Juan Pablo Angel slid Villa ahead but Richard Stearman slammed home to level after Momo Sylla's shot was saved.
Gareth Barry put Villa back in front from the penalty spot but Patrick Kisnorbo's header made it 2-2.
Barry had a second penalty saved before the end of normal time but Agbonlahor ensured Villa stay unbeaten this term.
Villa boss Martin O'Neill was given a hero's welcome by the home crowd who remembered his successful five-year spell in charge of the Foxes from 1995 to 2000.
That stint included winning the League Cup on two occasions in 1997 and 2000.
O'Neill's side have now gone 11 games undefeated since the start of the season but they were pushed all the way by a Leicester side who had not lost any of their previous seven matches.
Twice the Foxes came from behind and deservedly forced extra-time.
The game looked set for penalties before Agbonlahor made the most of Josh Low's failure to clear the ball and fired in a shot which was deflected high into the net.
Leicester: Henderson, Stearman, McAuley, Kisnorbo, Johansson, Maybury, Sylla (Kenton 101), Wesolowski, Welsh (Low 67), Hammond (Eric Odhiambo 76), Hume.
Subs Not Used: Douglas, Tiatto.
Booked: McAuley, Kisnorbo, Sylla, Maybury.
Goals: Stearman 42, Kisnorbo 85.
Aston Villa: Taylor, Hughes, Mellberg, Ridgewell, Barry, Agbonlahor, Osbourne (Bouma 107), Petrov, Davis (Agathe 91), Angel, Baros (Sutton 66).
Subs Not Used: Olejnik, Laursen.
Goals: Angel 5, Barry 45 pen, Agbonlahor 119.
Ref: M Clattenburg (Co Durham).