If all over-30s followed recommended guidelines on exercise, nearly 3,000 lives could be saved each year in Australia alone, say researchers.
More needs to be done to warn women of inactivity, as it outweighs other risk factors such as obesity, they say.
Lack of exercise is a known major risk factor for heart disease, says the British Heart Foundation.
A team at the University of Queensland, Australia tracked the health of more than 30,000 women born in the 1920s, 1940s and 1970s.
They found smoking had the greatest impact on women's heart disease risk below the age of 30.
However, as women got older and more gave up smoking, it was overtaken by physical inactivity as the dominant influence on heart disease risk.
Continuing efforts to encourage people to stop smoking were warranted, the researchers reported in the British Journal of Sports Medicine.
But they said greater effort were needed to promote exercise, which they describe as a "Cinderella" risk factor compared with obesity.
"We need a lot more effort to keep middle-aged women active and then keep them active into old age," Prof Wendy Brown, of the university's centre for research on exercise, physical activity and health, told BBC News.
"If you can do at least 30 minutes and preferably 45 minutes a day, you'll see huge improvements in your health and reduce your risk of heart disease by half."
According to NHS Choices, adults aged 19-64 years are recommended to do 150 minutes of weekly physical activity, or 30 minutes on five days a week.
Commenting on the study, Thembi Nkala, senior cardiac nurse at the British Heart Foundation, said physical inactivity was a known major risk factor for heart disease.
"Interestingly, this study shows its dominant influence on heart disease amongst women, and suggests a greater need to promote regular physical activity amongst this group," she said.
"It's important to remember that heart disease is linked to other factors such as smoking, obesity, high blood pressure and high cholesterol.
"It's essential to manage these too, as the more risk factors you have, the greater your chance of heart disease. "

After previously calling off the final four dates of their US leg, their two Canadian dates scheduled for Friday and Saturday will also not go ahead.
The band said doctors had told Rhodes he needed to continue to rest for at least another 72 hours.
They added that they were "devastated" their two-year tour had ended this way.
Doctors had previously told Rhodes the illness could be the result of a viral infection.
A statement on the band's website said: "Nick held out all week, hoping that he would be well enough to perform again for these final few shows, but unfortunately the medical advice he has been given today requires him to continue to rest.
"The band are devastated that what has been a triumphant tour for them, with many highlights, has ended this way - and know that the fans will be as disappointed as they are."
Earlier in the week, Rhodes said he felt "completely helpless over the situation and all I can do is apologise to our fans".
"We would never let them down unless it was totally unavoidable," he said.
The band had hoped to reschedule the dates, but it has not been possible because of the availability of the venues affected.
They said ticket holders for the cancelled gigs should apply for refunds at the point of purchase.
Duran Duran's All You Need Is Now tour has been under way since spring 2011.
They were forced to postpone a string of dates on the UK leg last summer after singer Simon Le Bon was treated for ongoing throat problems.

Media playback is unsupported on your device
19 June 2015 Last updated at 12:11 BST
Drone footage produced by Chester Unlocked has revealed hidden craftsmanship such as the Chester Imp carving high up in one of its windows.
Other iconic landmarks in the city will be filmed throughout the year.
Lack of exercise is the biggest risk factor for heart disease in women aged 30 and above, according to a study.
[NEXT_CONCEPT]
Duran Duran have been forced to cancel the remaining two dates of their world tour because keyboard player Nick Rhodes is still too ill to perform.
[NEXT_CONCEPT]
A history project has given a glimpse of Chester cathedral which has not been seen for centuries.