Patrick McLoughlin said Network Rail's five-year plan was being "reset" as it was "costing more and taking longer".
Schemes in the Midlands and Yorkshire would be put on hold, he told MPs.
Labour said it remained "concerned" about the future of the electrification of the Great Western line.
Sir Peter Hendy will become the new chairman of Network Rail in place of Gwynedd-born Richard Parry-Jones.
Mr McLoughin told MPs on Thursday that the planned programme of rail upgrades was being reviewed to "get it back on track".
"Electrification of the Great Western line is a top priority and I want Network Rail to concentrate its efforts on getting that right," he said.
But shadow transport secretary Michael Dugher said: "We remain concerned about the future of the electrification of the Great Western line."
Responding for the Welsh government, Economy Minister Edwina Hart said: "It's got to be kept on budget, it's got to be kept on time because ourselves and industry are desperate to ensure we have the electrification to Swansea.
"I'm hopeful that this is an absolute commitment with the government, but we will certainly be taking them to task if they deviate."
Network Rail chief executive Mark Carne told the BBC the challenges of delivering myriad improvement projects while still running a railway seven days a week were simply overwhelming.
Wales Office Minister Alun Cairns said he was "pleased to hear the strong commitment to south Wales electrification through to Swansea from the transport secretary today".
"This project will transform the lives of people in south Wales by attracting investors, linking businesses to suppliers and connecting people to jobs."
David Cameron had said during Prime Minister's Questions on Tuesday that ministers were "absolutely committed" to the London-Swansea upgrade following a newspaper report that some projects would be shelved.
Electrifying the Great Western line is "a top priority", the transport secretary has said, as he announces a rethink of a £38bn programme to overhaul Britain's railways.