The successor to Jacques Bihozagara as Rwanda's ambassador to Belgium has called the death an assassination.
No official reason has been given for his death.
Relations between the two neighbours have been strained recently, amid claims Rwanda was backing rebels seeking to overthrow Burundi's president, something Rwanda denies.
Africa Live: BBC news updates
Mr Bihozagara was dead within minutes of arriving at the prison hospital, according to fellow prisoners.
A Rwandan foreign ministry official told AFP news agency that he had been illegally detained and urged the authorities to clarify this sudden death, "which raises many questions".
Mr Bihozagara was arrested last December by Burundi's intelligence services on suspicion of spying for his country.
He was a retired ambassador and former youth minister.
He lived in Burundi before the Rwandan genocide in 1994 and after his retirement, he visited the country regularly on business, reports the Rwandan newspaper the New Times.
Rwanda and Burundi have a difficult relationship.
In February, Rwanda was accused of training Burundian rebels in its refugee camps and subsequently announced that it would try to relocate all Burundian refugees to other host counties.
At least 240,000 people have fled Burundi in the last year.
Burundi has been hit by unrest since President Pierre Nkurunziza decided to stand for a third term last April.
A former Rwandan minister has died in jail in Burundi nearly four months after being arrested for espionage.