BBC London has found 94% of hospitals in the capital contain asbestos.
About 1,000 people have died from mesothelioma since 2011 in London, seven of which were doctors and nurses.
The Unite union and a leading lawyer in asbestos claims both say the figure is increasing, but the Health and Safety Executive said hospitals were safe.
White asbestos, the type used in hospitals to protect piping, has been deemed safe as long as it is not disturbed.
But Jerry Swain, acting national instructor for Unite's construction centre, and lawyer Isobel Lovett, who has dealt with asbestos cases for 17 years, have both described the number of people who are developing mesothemelioma as a "ticking time bomb".
Source: Doctor Peter Szlosarek, consultant oncologist at Barts Hospital who is studying mesothemelioma
"White asbestos in hospitals is still a danger - there's no safe form of asbestos," said Ms Lovett. "There's no safe level of dust to which you can be exposed. All asbestos dust, once breathed in, presents a hazard."
She added that while the source of white asbestos remains in hospitals, there was always a danger that someone could inadvertently disturb it in some way and release the dangerous fibres.
Mr Swain added: "It's very hard to imagine a killer being safe. Asbestos is a known killer, people are dying from it. The only safe asbestos is asbestos that's been removed.
"If we're going to leave asbestos in places, we have to be aware that we are taking a conscious decision and that people will die."
He said a plan was needed to eradicate it because some workers may be unaware of the dangers.
Juliet Cohen, whose husband Andrew Lawson was an anaesthetist and specialised in intensive care medicine and chronic pain management, claimed he was exposed to asbestos walking through the underground tunnels linking hospital buildings at Guys and St Thomas Hospital when he trained there between 1976-82.
He died in 2014, seven years after being diagnosed with mesothemelioma.
"At the time there was asbestos lagging in poor repair in these underground tunnels linking hospital buildings," she said.
"It means that every single person walking through was exposed on a daily basis. The risks of asbestos were well known at the time, that is something that is completely indefensible."
She said her husband was contacted "on a daily basis" by doctors and nurses who were concerned about their exposure.
Ms Cohen added that all the health trusts knew about the dangers of asbestos.
"They simply failed to take enough action to remove the risk for people who work there," she added.
The hospital said the asbestos in the basement area which Dr Lawson was exposed to was removed in the 1990s.
The government has not yet responded to the BBC's requests for a comment.
The number of people who could contract cancer from asbestos poisoning in London's hospitals is a "ticking time bomb", it has been claimed.