Cabinet minister Peter Hain has said he thinks the US-run detention camp at Guantanamo Bay should be shut down.
Mr Hain's comments follow a critical UN report on the camp
A UN report has said aspects of the regime at the camp amounted to torture.
The Northern Ireland secretary told BBC One's Question Time: "I would prefer that it wasn't there and I would prefer it was closed."
Asked about the comments Tony Blair said: "I've said all along... that it [the prison] is an anomaly and sooner or later it's got to be dealt with."
Mr Hain said he thought the prime minister shared his view about closing Guantanamo Bay.
But Mr Blair did not expand on what he has previously told MPs.
Meanwhile, three British residents held at Guantanamo Bay have won permission to seek a High Court order requiring the UK to petition for their release.
A judge said claims of torture at the camp meant the government might have an obligation to act on their behalf.
But there were "formidable arguments" against Bisher al-Rawi, Jamil el-Banna and Omar Deghayes's case, he added.
The judge Mr Justice Collins remarked during the hearing on Thursday that the US' idea of torture "doesn't appear to coincide with that of most civilised countries".
And, asked on BBC One's Question Time whether closing the camp was now government policy and whether the prime minister ought to make that clear to the US administration, Mr Hain said: "It's obvious if you don't think it should be done it should be closed, isn't it?"
Asked if Tony Blair agreed, Mr Hain said: "I think so, yes."
The chairman of the Commons foreign affairs select committee, Mike Gapes, said there was a feeling among MPs that a "more forceful statement" should be made about the camp.
Mr Gapes welcomed Mr Hain's comment, and when asked on BBC Radio 4's Today programme why he thought Mr Blair was not being "bolder".
"I suspect it's part of a general approach to speak quietly to the Americans and not make big public statements," added Mr Gapes.
Shadow foreign secretary William Hague, in Washington to build bridges with the White House, warned the US its reputation was being damaged by reports of abuse at the camp.
"Reports of prisoner abuse by British and American troops - however isolated - and accounts, accurate or not, of the mistreatment of detainees at Guantanamo and extraordinary rendition flights leading to the torture of suspects, have led to a critical erosion in our moral authority," Mr Hague said in a speech to a Washington think tank on Thursday.
"This has resulted in a loss of goodwill towards America which could be as serious in the long-term as the sharpest of military defeats."
The UN report says the US's treatment of detainees, some of whom have been held for more than four years, violates their rights to physical and mental health.
It expresses concern at the use of excessive force during transportation and force-feeding through nasal tubes during hunger strikes, which it says amounts to torture.
Responding to the report, UN Secretary General Kofi Annan, said that "sooner or later" there will be a need to close the camp.
He said he hoped the US Government would decide to do this "as soon as possible".
But the US Government said the document was "a discredit to the UN".
White House spokesman Scott McClellan said investigators had failed to examine the facts and that their time would be better spent studying other cases.