Since April, convicted criminals in England and Wales have had to pay a charge of between £150 and £1,200 towards the cost of their case.
But Mr Gove said "while the intention behind the policy was honourable in reality that intent has fallen short".
MPs had called for it to be axed and the Magistrates Association said the decision was "tremendously welcome".
The charge is paid on top of fines, compensation orders and defendants' own legal charges, and is higher for those convicted after pleading not guilty.
It is set according to the type of case, with the minimum charge for magistrates' courts and the maximum level for crown court cases.
In March, the then Justice Secretary Chris Grayling said the fee would ensure that criminals "pay their way".
But MPs on the Justice Committee published a report in November in which they said the fee, which is not means-tested, created "serious problems" and was often "grossly disproportionate".
The cross-party group's chairman, Conservative MP Bob Neill, said they evidence they had received raised "grave misgivings" about the fee's benefits and whether it was "compatible with the principles of justice".
He said it created "perverse incentives - not only for defendants to plead guilty but for sentencers to reduce awards of compensation and prosecution costs".
Mr Gove, Mr Grayling's successor as justice secretary, announced it was being dropped, during a meeting of the Magistrates' Association Council.
The Magistrates' Association's national chairman, Malcolm Richardson said: "This is an enormous success for the MA (Magistrates' Association) but most importantly for justice in our criminal courts system.
"In my 26 years as a magistrate I have never seen such a powerful reaction from my colleagues on the bench and Michael Gove's announcement is therefore tremendously welcome."
Justice Select Committee chairman Bob Neill said it was "good" Mr Gove was "willing to respond so swiftly to take account of the inquiry we carried out and the evidence submitted to us".
"We will watch with interest the review which Michael Gove has announced into the whole panoply of financial impositions on offenders," he added.
The Howard League for Penal Reform - which has called for the fee policy to be suspended - also welcomed the news, saying the charge was "simply unfair".
Its chief executive, Frances Crook said: "This is a victory for justice. It augurs well for changes being made to the courts and penal systems, which both need radical reform."
Meanwhile, Labour said it had warned from the start that the "mandatory nature of the fee could lead to miscarriages of justice" and welcomed that it "has finally been scrapped".
Mr Gove told Parliament in November that the charge was generating revenue and helping to "ensure that the taxpayer is not the first port of call for supporting the way in which our courts operate, but it is important that we balance all the criteria in making a judgment on the review of the charge".

The 25-year-old Spain international joined the Potters from Barcelona for an undisclosed fee in July 2014.
Bojan, who missed the second half of last season with a knee injury, has scored 10 goals in 41 games for Stoke.
Chief executive Tony Scholes said: "Bojan has made no secret of the fact that he now regards Stoke as his home in a football sense."
Bojan said: "I am very happy and motivated. Stoke City gave me the opportunity to play in the most competitive league in the world, and I have only words of gratitude for their trust and for the way they have treated me.
"Mark Hughes convinced me to come to Stoke, he has helped me and showed his trust in me from the beginning.
"He followed closely the recovery process from my injury and there is no doubt I have signed an extension of my contract thanks to him."
The criminal courts charge is to be scrapped from 24 December, Justice Secretary Michael Gove has announced.
[NEXT_CONCEPT]
Stoke City forward Bojan Krkic has signed a new four-and-a-half-year contract with the Premier League club.