The Conservative MP represents Dumfriesshire, Clydesdale and Tweeddale which borders with Cumbria.
He has pledged to work with the Scottish government to look at taking the line beyond Tweedbank.
The SNP has already given a commitment to a feasibility study into looking at extending the track.
Earlier this year First Minister Nicola Sturgeon said it made sense to look at the possibility.
In a speech in Edinburgh, Mr Mundell said: "Scotland needs good transport links to thrive.
"During the election, the first minister stated her commitment of exploring the feasibility of extending the Borders Railway to Carlisle.
"Patently, such a development would require cross-border working and partnership."
He said he was ready to work with the Scottish government on the issue, as it developed its "plans and thinking".
He added: "I think it is a realistic prospect.
"It obviously requires considerable work in terms of ensuring that the route is identified.
"There are complexities because, obviously, it crosses a border and the legislation and rules required to create a railway in England are slightly different from those in Scotland.
"But I think everyone is of a mind that it should be done and the funding obviously is identified."
He said it could make a "major difference" for communities like Hawick.
"It's also got huge potential in the south to take traffic off the roads. I have committed to play my part," he said.
Extending the Borders Railway to Carlisle is "a realistic prospect", according to Scottish Secretary David Mundell.