Patients have not seen any improvement in the care they get under the new consultant contract, a watchdog says.
The contract set out new working arrangements
The National Audit Office said despite pay rising by 27% to £110,000, doctors were not providing more flexible care or spending more time with patients.
But the watchdog blamed ministers and health managers for the way the 2003 deal was implemented in England, and said it cost £150m more than expected.
The government said the contract still had "considerable potential".
Over the first three years of the contract, as well as seeing the jump in pay - which does not take into account private earnings - specialist doctors also saw their hours cut.
These changes were expected, as consultants have always argued they were under-paid for the hours they worked.
But at the time the government said it was a "something for something" contract and would lead to increased productivity and better patient care.
The contract - the biggest shake-up in the way consultants worked since the NHS was formed - was designed to free them up to spend more time with patients and provide more flexible services, such as evening clinics.
The National Audit Office (NAO) report said it had failed on both counts.
Since the contract started, consultant numbers have risen by 11% to 31,990, but the number of patients seen has only gone up by 4%.
The NAO also said the contract has cost at least £715m.
It added that the government had underestimated the amount of work consultants did, and NHS trust managers had failed to keep within budget.
The watchdog said NHS trusts needed to improve job planning to improve access to service, while consultants should become more engaged with managers.
Karen Taylor, director of health at the NAO, said: "Patients are not seeing consultants work any differently, but if trusts managed job planning better, there is potential for this to change."
A spokeswoman for the Patients Association said: "The consultants' contract was a quick fix.
"Those responsible for the negotiations now have a duty to the public ? shareholder of the NHS - and patients - the customers to ensure value for money."
Liberal Democrat health spokesman Norman Lamb said: "This is yet further proof of the government's incompetent stewardship of the NHS."
And shadow health secretary Andrew Lansley said there was "no evidence of improved productivity".
Dr Mark Porter of the British Medical Association's consultants committee told BBC Radio 4's Today programme: "The vast majority of consultants are continuing to provide the professional care to their patients that they want as they always have."
But he said there were tensions about how that was reflected in the number of hours trusts recognised they worked.
"A small number of consultants have seen real reductions, but the majority are continuing to do their professional job."
Alastair Henderson, deputy director of NHS Employers, said the new contract did provide a good framework.
But he added: "I think it will continue to work better in the future."
Health Minister Lord Hunt said: "There is considerable potential for the consultant contract to further improve the management of consultant time and deliver full value for money to the NHS."
The criticism of the consultants' contract comes after the government was also attacked for the new contract for GPs which, since it started in 2004, has seen GP pay break through the £100,000 a year barrier.