They were among more than 1,000 people who invested in the Ingenious film financing schemes in the hope of securing tax relief.
Ingenious, which helped produce movies including Avatar, qualified for tax breaks designed to support the UK film industry.
But HMRC said Ingenious claimed relief on artificial losses from its films.
That meant the schemes were not legitimate investment opportunities but actually a means of avoiding tax.
On Wednesday, a tax tribunal upheld a 2016 decision to recoup the avoided tax, ruling that the incentives were "not allowable deductions".
It means that 1,400 people - including celebrities such as Gary Lineker, Bob Geldof and Ant and Dec - now face big bills.
Some £420m of tax was avoided, but with interest included the total owed will be closer to £700m.
An HMRC spokesperson said: "We are pleased that the tribunal has agreed with us that the vast majority of what was claimed in tax relief by Ingenious investors was simply not due."
A spokesman for Ingenious said it would be appealing the ruling: "It is wholly unsatisfactory that the tribunal reached this decision with 'misgivings and reluctance."
Those who used the Ingenious scheme had to invest at least £100,000 and were promised generous tax breaks on any losses incurred.
Many subsequently claimed that they were poorly advised about the schemes and had been unaware of the risks.
HMRC has previously won cases against another film tax avoidance scheme called Icebreaker.
In August 2016, the Tax Tribunal ruled members had claimed tax relief on losses many times higher than the actual amount they invested in the partnerships. The total tax at stake was £134m.
Celebrities including David Beckham and Wayne Rooney have lost a legal bid to overturn a £700m tax bill.