After a bright start for the home side, Hearts dominated the first half and only three good saves from goalkeeper Zander Clark kept them at bay.
Cummins pounced shortly after the break to divert a miss-hit Liam Craig shot with Hearts claiming for offside.
Callum Paterson, with a header against the bar, came closest to an equaliser.
But it was St Johnstone who secured their second win over Hearts this season - having beaten the Edinburgh side in the League Cup - to move into third place behind their opponents on goal difference.
To add to Hearts' disappointment, head coach Robbie Neilson was sent to the stand as his side pressed for that elusive leveller.
And Celtic now have a chance to extend their lead to five points with a game in hand when they travel to face Inverness Caledonian Thistle on Sunday.
The Hearts head scratching will go on for some time after what was a dominant display, but they had one man to blame for their defeat - goalkeeper Clark.
Clark has been standing in for the injured Alan Mannus, but his display was such that manager Tommy Wright will have a tough decision to make when his number one is fit and ready for action.
The 24-year-old pulled off two top-class, close-range saves in the first half - stage first from Faycal Rherras and the second from Sam Nicolson, who should have done better from six yards out.
As Hearts piled forward in the second half, he also denied Igor Rossi and then tipped a Tony Watt free kick over the bar.
Clark walked off the pitch to thankful back slaps from his team, respectful handshakes from the visitors and the fans singing his name.
St Johnstone under Tommy Wright have become a very difficult team to beat and even harder when they go ahead.
Every player knows his job and their organisation and work rate is consistent and impressive - and against Hearts it was no different.
Hearts' Robbie Neilson started with Sam Nicolson and Jamie Walker on the wings in a bid to stretch the match and flood the box with crosses.
In that sense, his game plan worked, but the Saints defence fought for every ball, challenged every cross and chased every lost cause.
After going ahead, they were doggedly determined to stay there and it was a frustrating afternoon for those in maroon.
The build-up to the match was dominated by Wright's comments about Hearts players going to ground too easily - and he said in the minutes before kick-off that he stood by his stance.
The Irishman was left looking rather sheepish though after two of his players were booked for simulation.
Goalscorer Cummins, in the first half, and substitute Chris Kane were both cautioned by referee John Beaton.
Neilson resisted the temptation to gloat, although he had his own problems after being sent to the stand.
Media playback is not supported on this device
St Johnstone manager Tommy Wright: "We can get results like that through sheer determination and quality goalkeeping and a desire not to let the ball go in the net from everybody in the team, hard work, the odd wee bit of quality, probably not as much as I would like.
"A hard-fought win and it is a perfect one for a manager - a clean sheet and getting a goal at the other end.
"I've got two number one goalkeepers - that's rare at football clubs and, having been a goalkeeper, I know that.
"Over the last six months, Zander has improved immensely and I don't think there is a better young goalkeeper in Scotland and I think he should be in and around that Scotland squad."
Hearts head coach Robbie Neilson: "I spoke to the players after the game and said we played some fantastic football and created some wonderful chances.
"But we didn't take them and got done with a sucker punch, but I'd rather play like that and be entertaining and exciting and create chances than go direct and hope to nick a goal.
"I thought we played really well to a man, but we just couldn't get the win our play deserved.
"The St Johnstone goalkeeper has some wonderful saves and probably had the game of his life."
Match ends, St. Johnstone 1, Heart of Midlothian 0.
Second Half ends, St. Johnstone 1, Heart of Midlothian 0.
Attempt missed. Callum Paterson (Heart of Midlothian) right footed shot from the centre of the box misses to the left.
Attempt saved. Christopher Kane (St. Johnstone) right footed shot from the centre of the box is saved in the centre of the goal.
Corner,  St. Johnstone. Conceded by John Souttar.
Liam Craig (St. Johnstone) is shown the yellow card.
Christopher Kane (St. Johnstone) wins a free kick in the defensive half.
Foul by Igor Rossi (Heart of Midlothian).
Substitution, St. Johnstone. Blair Alston replaces Steven MacLean.
Attempt missed. Sam Nicholson (Heart of Midlothian) right footed shot from the right side of the six yard box is close, but misses to the right.
Callum Paterson (Heart of Midlothian) hits the bar with a header from the centre of the box following a corner.
Corner,  Heart of Midlothian. Conceded by Steven Anderson.
Corner,  Heart of Midlothian. Conceded by Murray Davidson.
Foul by Steven MacLean (St. Johnstone).
Igor Rossi (Heart of Midlothian) wins a free kick in the attacking half.
Attempt saved. John Souttar (Heart of Midlothian) header from the centre of the box is saved in the top centre of the goal.
Foul by Christopher Kane (St. Johnstone).
Perry Kitchen (Heart of Midlothian) wins a free kick in the defensive half.
Foul by Christopher Kane (St. Johnstone).
Igor Rossi (Heart of Midlothian) wins a free kick in the attacking half.
Paul Paton (St. Johnstone) wins a free kick in the defensive half.
Foul by Arnaud Djoum (Heart of Midlothian).
Christopher Kane (St. Johnstone) is shown the yellow card.
Attempt missed. Sam Nicholson (Heart of Midlothian) left footed shot from outside the box misses to the left.
Substitution, Heart of Midlothian. Robbie Muirhead replaces Don Cowie.
Attempt missed. Callum Paterson (Heart of Midlothian) right footed shot from the centre of the box is too high following a corner.
Corner,  Heart of Midlothian. Conceded by Steven Anderson.
Attempt blocked. Callum Paterson (Heart of Midlothian) right footed shot from the right side of the box is blocked.
Attempt missed. Liam Craig (St. Johnstone) left footed shot from outside the box is close, but misses to the right.
Richard Foster (St. Johnstone) wins a free kick on the right wing.
Foul by Igor Rossi (Heart of Midlothian).
Substitution, St. Johnstone. Christopher Kane replaces Graham Cummins.
Hand ball by Igor Rossi (Heart of Midlothian).
Corner,  Heart of Midlothian. Conceded by Steven Anderson.
Igor Rossi (Heart of Midlothian) is shown the yellow card.
Steven MacLean (St. Johnstone) is shown the yellow card.
Foul by Graham Cummins (St. Johnstone).
Igor Rossi (Heart of Midlothian) wins a free kick in the attacking half.
Foul by Steven MacLean (St. Johnstone).
John Souttar (Heart of Midlothian) wins a free kick in the attacking half.
Graham Cummins gave St Johnstone a hard-worked victory over Hearts to join the visitors two points behind Scottish Premiership leaders Celtic.