A hole was smashed in the front of the Co-op store on Main Street, Great Glen, Leicestershire in the early hours of Tuesday morning.
The fork-lift loader was left blocking the road and the cash machine itself was found nearby.
Police said four men were seen trying to load the machine on a truck and have appealed for more information.
Witnesses said glass, bricks and metal were left scattered on the road with shop's alarms sounding continually.
The truck was later recovered in a street nearby.
Officers said they believe the gang failed to make off with any money.
A supermarket was badly damaged when a hydraulic lifter was used to rip a cash machine from a wall.