Think-tank Reform Scotland said the route had been left with limited potential for expansion.
It said the Edinburgh to Tweedbank line had shown thinking was "too small" and planning "too short-term".
The Scottish government said the railway had proved a huge success and expected more investment in future.
Reform Scotland released written evidence it had sent to Transport Scotland's rail infrastructure strategy consultation.
It has been submitted on its behalf by advisory board member Tom Harris, a former Labour transport minister in the UK government.
He makes a range of suggestions including the need for "bigger thinking" in the long-term vision for Scotland's railways.
Mr Harris singled out the Borders Railway for criticism with the construction of bridges to accommodate only single-track width as an example of limits placed on future expansion.
He said: "We are in danger of missing the bigger picture when it comes to discussing rail in Scotland.
"We talk in very narrow terms about ScotRail and its operation, but what we actually need is an injection of some creative, strategic thinking so that we can give the Scottish people a rail system built for the future and one to be proud of. "
He called for the creation of a Scottish Rail Infrastructure Commission to examine what "ambitious transformational projects and new railway lines" were needed to boost the Scottish economy and "transform our connectivity as a nation".
"The Borders Railway showed us that there is an appetite for new railways in Scotland, but it also showed us that our thinking is too small and our planning too short-term," he said.
"That ship has now sailed, but we must learn the lessons from it, think big, and plan long, and that is why the need for a commission is now critical."
A Scottish government spokesman said it welcomed Reform Scotland's comments.
"The think-tank has raised issues about structure and vision," he said.
"Whilst there will be different views on structures to realise the vision for Scotland's railways it is clear that the ambitions for our railway to be a driver of economic growth and a positive, integral part of our social fabric is an ambition that is shared right across Scotland."
He said the reopening of the Borders Railway had been a "cause for celebration" in the Borders and Midlothian.
"We were always confident that, in time, it would deliver major economic opportunities and attract new investment, so it is pleasing to see strong evidence that visits and spending are up significantly as a direct result of the new railway," he said.
"With over 1.2m passenger journeys in its first year alone, additional seats on peak time services and more improvements and initiatives in the pipeline, we look forward to seeing even more investment and interest in the area in the months and years ahead."

Valdes, 32, has signed an 18-month contract at Old Trafford.
De Gea, 24, won United's player of the year award last season.
"Nobody is happy to be number two and Victor has to have ambition to beat David De Gea - but it shall be very hard for him," said Van Gaal.
Valdes had been training at United as he recovered from a serious knee injury.
De Gea has 18 months remaining on his present deal with United after signing for the club in 2011, but he has recently been linked with a move to Real Madrid.
However, the United boss says he will extend the player's terms with the club rather than succeed the 33-year-old Iker Casillas at Real.
When asked if he thought De Gea would stay, Van Gaal said: "I think so, yes. There is no problem for him to sign."
The Dutchman also does not see an issue with having two full Spain internationals in his squad fighting for one position.
"My ambition is that I have the best players who can collaborate with each other to form the best team in the world," he said.
"When we invited Victor in to have his rehabilitation at Manchester United, we did it for him and ourselves, so I could see if his quality is also still the same. He has confirmed both."
Valdes' arrival on a full-time basis is likely to mean the exit of previous back-up goalkeeper Anders Lindegaard.
Unbeaten in 11 games, United play Southampton with their manager able to pick from a virtually fully fit squad for the first time this season.
"We only have one player injured. Can you believe it?" said Van Gaal.
"I have to select, for the first time in my career at Manchester United, the 11 players and the 18."
Van Gaal confirmed England winger Ashley Young would be missing for between four and six weeks with a hamstring injury.
The Scottish government has defended the Borders Railway after claims the infrastructure vision for the project was "short-sighted".
[NEXT_CONCEPT]
Manchester United manager Louis van Gaal says he signed Victor Valdes knowing the former Barcelona goalkeeper will not be happy being back-up for fellow Spaniard David De Gea.