Rail passengers faced 800,000 delays on Britain's trains in 2006/7 and were poorly informed about them, according to the National Audit Office (NAO).
Delays on the railways added up to 10,000 lost days
The watchdog said 14 million minutes were lost by late running trains, costing the public £1bn in lost time.
The NAO report also highlighted shortcomings in contingency plans put into place when delays occur.
Shadow Transport Secretary Theresa Villiers described the disruption as "too much misery for passengers".
In the loop
NAO Comptroller and Auditor General Tim Burr said: "The rail industry has made progress in keeping trains moving... but, when incidents happen, passengers should get better information about what is happening."
The chairman of the House of Commons Public Accounts Committee, Edward Leigh, said there were still too many delays on Britain's railways.
"Not only is this frustrating, it is also expensive - costing over £1bn a year in lost time for passengers.
"At a time when fares are on the increase across the country, passengers have the right to expect that delays are kept to an absolute minimum."
He called for passengers to be given information about a delay as promptly as possible, ideally within two minutes.
"There is nothing more infuriating than being stuck on a train or a platform with no information about the cause of delay and how long it will take to resolve," Mr Leigh added.
Theresa Villiers said passengers had a right to better levels of service following above-inflation rises in rail fares.
"This report shows once again how important it is that Network Rail finally realise that their customer is the travelling public - not the Fat Controller in Whitehall", she said.
"Passengers deserve better but, because of the fundamentally flawed way Gordon Brown set up Network Rail, it is accountable to no-one and can too often get away with impunity."
Liberal Democrat transport spokesman Norman Baker said that, while some delays were inevitable, more action was needed to ensure a consistent service.
"If the railways are to continue to grow, passengers must be offered a good and reliable service, rather than constantly being faced with delays and cancellations", he said.
The Department for Transport welcomed the report.
A spokesman said: "A reliable railway is essential for passengers and for the economy.
"Performance has seen a 10% improvement on five years ago, but more can and will be done to improve this".
The government's spending watchdog also criticised the way Network Rail and the emergency services tackle the most common kinds of delay, including trespassing on the railways and vehicles hitting train bridges.
Among its other findings the NAO said:
- There were shortcomings in the way that passengers were handled when incidents occur
- Network Rail staff felt that on occasion local police force practices actually endangered the safety of passengers on delayed trains and in overcrowded stations
- Inadequate training of emergency services staff meant they weren't always aware of who to contact during incidents or how to work safely on the railways.