The Northern Ireland striker headed home on the hour mark as Ian Cathro's side struggled at Borough Briggs.
Dunfermline, having played a game more, top Group B after a 9-8 penalty shootout win over East Fife following a goalless draw.
The shootout victory meant the Pars picked up two points with East Fife taking one.
After 17 penalties without a miss, Ben Gordon was the unlucky man to fail with his spot-kick.
Hearts head coach Cathro was happy to see summer signing Lafferty open his account for the club.
"We anticipated it would be difficult, it's not an easy place to come and play," he told BBC Scotland. "They defended well and we had to be patient.
"I'm pleased that we stayed calm and that Kyle (Lafferty) got a goal. It was a good finish."
In Group A, Inverness Caledonian Thistle held on to top spot despite needing a shootout following a 0-0 draw away to Stirling Albion.
Liam Polworth and Iain Vigurs were the only ones to score in the shootout to hand John Robertson's men a 2-0 win on penalties and the bonus point.
In the derby between Brechin City and Forfar Athletic, Mark Millar took the game to spot-kicks with a late penalty after Isaac layne had given Brechin a second-half lead.
Brechin moved into third place, one point behind Falkirk, after Kalvin Orsi's winning penalty earned the bonus point.
Inverness Caley Thistle manager John Robertson blamed a lack of a killer instinct for his side failing to earn all three points.
"In terms of the ball, we dominated most of the match," said Robertson. "We knew Stirling would be at it and they were, all credit to Dave Mackay and his team.
"We lacked belief in the first half, had great chances to score and didn't take them. Second half we said up the tempo and get at them, which we did. But we just couldn't find that killer instinct.
"The more it went, the more frustrating it was. I thought we were going to be here until midnight with the penalties. Two scored out of eight, that's incredible. Two of our guys missed the target and for professional players that's bad technique."

Koy Bentley was fatally wounded on the fifth floor of a block of flats in Watford on Monday afternoon.
Hertfordshire police initially said the death was "suspicious". No arrests have yet been made.
Det Insp Pushpa Guild said: "Extensive inquiries are continuing at this time to establish the exact circumstances surrounding Koy's death."
An appeal for information has been launched.
Kyle Lafferty scored on his debut to give Hearts a 1-0 win over League Two Elgin City in the League Cup.
[NEXT_CONCEPT]
The death of a 15-year-old boy found stabbed in the chest is being treated as murder, police have confirmed.