Jamal al-Harith reportedly received £1m from the British government after being freed from Guantanamo Bay in 2004.
Lord Carlile said the payment was wrong as al-Harith was "plainly a terrorist".
Former Prime Minister Tony Blair has defended his government's decision to free him from Guantanamo.
Al-Harith, who was 50 and from Manchester, was originally known as Ronald Fiddler.
He took the name Jamal al-Harith when he converted to Islam, but was known most recently by the nom-de-guerre Abu-Zakariya al-Britani, given to him by so-called Islamic State.
Who are Britain’s jihadists?
Al-Harith was seized by American forces in Pakistan in 2001, before being sent to Guantanamo Bay - a US prison in Cuba for terrorist suspects.
US interrogators found he provided useful information about the Taliban's methods, and he was released after two years.
He later joined IS and blew himself up at an Iraqi army base in Mosul this week.
Lord Carlile - who reviewed terror laws from 2001 to 2011 - told BBC Radio 4's Today programme: "It [the compensation] should never have been paid.
"There was absolutely no merit in paying him a penny, because plainly he was a terrorist."
He said he believed the settlement was paid to avoid disclosure in court of security service activities.
A Downing Street spokesman declined to answer questions about the reported payout, on the grounds it was an intelligence matter.
But Mr Blair released a statement accusing the Daily Mail of "utter hypocrisy" after it ran a story about al-Harith on Wednesday headlined: "Still Think He Wasn't A Danger, Mr Blair? Fury at Labour government's £1m compensation for innocent Brit".
He said the man's release in 2004 had "followed a Parliamentary and massive media campaign led by the Daily Mail... and strongly supported by the then Conservative Opposition".
The former PM continued: "He was not paid compensation by my government. The compensation was agreed in 2010 by the [coalition] government..."
Lord Blunkett, who was home secretary at the time of al-Harith's release, said he had never campaigned for his return, but "fully accepted that the situation of British citizens held without trial there, was unsustainable and legally and morally indefensible".
The government in 2004 had "acted responsibly" he said, adding that "public controversy" at the time had been about whether enough was being done to release detainees "and not the wisdom of providing balanced reassurance".
Lord Blunkett said those returning from Guantanamo Bay were kept under surveillance and monitored by the security services.
Jack Straw, who was foreign secretary in 2004, said he "never regarded" al-Harith as innocent "and neither Mr Blair nor I ever said that he was innocent".
"We judged that the risk was not so great as to prevent his release.
"Whenever you're making decisions about the release of prisoners you have to make a judgement, and sometimes those judgements are not borne out by events."
Leon Jameson, al-Harith's older brother, says they last spoke two years ago on the phone, before he went to Syria.
Mr Jameson described his sibling as "fun" when he was growing up and "always helping other people".
When asked about his brother's suicide bombing he said: "I can't actually commend him about it because it isn't right, but he's done it. It's something he believes in, so I'll leave that with him.
"He did what he could for other people, which is what he used to always be like.
And he said "it had been a struggle" for his brother ever since Guantanamo Bay. "If he didn't even listen to his wife, none of us could have really changed his mind."
A British IS fighter who carried out a suicide attack in Iraq should never have received a penny in compensation, a former independent reviewer of UK terrorism legislation has said.