World's first VR-based surgery goes live

Shafi Ahmed, a cancer surgeon at Barts Health NHS Trust, will live-stream a surgery using a virtual reality headset in an effort to make healthcare 'more equitable' and provide intensive training for medical students. Using a virtual reality headset will add educational value and provide a more immersive experience for the viewer as, unlike Google Glass, students will be able to hear what is being said during the surgery allowing Ahmed to teach along the way. In the future, this technology could be paired with devices that replicate touch and feel to create a more realistic virtual reality.

Stretched out on a table in a large, bright operating theatre at the Royal London hospital, a patient is awaiting Shafi Ahmed’s first incision in a procedure that will remove cancerous tissue from his bowel. Around the table a team dressed in blue scrubs and face masks are gathered, exchanging the odd word, while cumbersome machines bearing bundles of wires hum gently in the background. Everyone is focused on the task in hand, getting ready to play their part. Except me.

Scrubless and without so much as a scalpel to pass to the surgeon, I am a mere spectator to this intricate event, a bystander gazing around the room in fascination while others labour at a life-changing task.

Not that the surgeons are bothered. Because although I feel like I am standing at the edge of the operating table, in reality I am sitting in my office chair.

On Thursday afternoon I witnessed the world’s first operation to be streamed live in 360-degree video, allowing medical students, trainee surgeons and curious members of the public like me to immerse themselves in the procedure in real time via the Medical Realities website. A one-minute delay was incorporated into the broadcast in case of any complications in the surgery.

A cancer surgeon at Barts Health NHS Trust, Ahmed said before the operation that he believed the approach could make healthcare more equitable, improving the training of surgeons worldwide. With internet connections becoming better, smartphones getting cheaper and only a pair of lenses and some cardboard needed to make a virtual reality headset, the costs, he said, paled in comparison to the expense of students travelling abroad to train. “It is actually quite cost-effective.”



Shot using two 360-degree cameras and a number of lenses arranged around the theatre, the operation could be viewed through the “VR in OR” app, using a virtual reality headset that can be paired with a smartphone. Those who did not have a headset could watch the feed live online.



It takes a while to get the app up and running so while I wait for the VR experience to start I watch the procedure begin via the website.

The lights are dimmed and, wielding an intimidating device, Ahmed begins to remove a hernia. “This is called a harmonic scalpel,” he says as he gets to work.

A hush descends, punctured only by beeping. Peering down I spot some odd-looking scissors I hope no one will ever use on me. Two large screens on either side of the table show views from the camera inside the patient – a device that resembles an enormous knitting needle. The team prepares to tackle the cancer. “OK let’s have a look, here we go,” says Ahmed.

Fortunately the app boots up. And I am in the room too. “There is a tumour just here,” says Ahmed. “This is what cancer looks like in reality.”

While videos showcasing surgical procedures have been around for years, Ahmed believes the new approach is more than a mere gimmick. The technology, he has argued, brings a valuable new feature to education, allowing viewers to focus not just on what the surgeon is doing, but also on what other members of the team are up to: “There will be noise, there will be the immersive factor – so that will add different layers of educational value.”



George Hanna, professor of surgical sciences at Imperial College, London was cautiously optimistic about the benefits of the approach. “If this technology allows the transfer of knowledge and skills [over] a wider range and in an easier way that would be very beneficial,” he said.

But he was quick to add that, compared with existing approaches for sharing scenes from the operating theatre, the new technology offered more of an upgrade than a revolution. “It is a good video and wide broadcast with interactive [opportunities],” he said, stressing that the operation itself was real rather than virtual.



Back in the virtual world, there is a slight hitch – the quality of the video on my phone is good, but not good enough to see the video screen clearly. It looks like they are fishing about in a swamp of slimy undercooked sausages: I suspect a medical student would glean rather more. Frustratingly, 360-degree video is just that – you can look around the room in every direction, but you can’t move for a better view.

The procedure continues. Despite my loathing of all medical TV dramas, I am hooked. Suddenly the entire theatre goes dark, except for a spotlight. Someone is making an incision. They pull out a pink, fleshy mass. “Scissors,” says someone. A bundle that I assume is the tumour is removed and dropped into a bowl. The operation has been a success.

It was not the first time Ahmed had led the way in embracing modern technology in healthcare. He co-founded the healthcare company Medical Realities, which streamed the operation in partnership with Barts Health and 360-degree video experts Mativision.

Ahmed said he believes virtual reality, augmented reality and games could all play a role in training medical students. Two years ago he streamed a live operation using the “augmented reality” system, Google Glass, allowing viewers to see the procedure from a surgeon’s point of view.

But the new 360-degree video, Ahmed said, offered a new approach, allowing users to see beyond what the surgeon was looking at. Among future developments he has envisaged, Ahmed said he was keen to add graphics to the raw footage to provide additional information during the operation, as well as taking questions from those viewing the procedure.

“[During an operation] I am teaching people, talking to them, there is communication going on – so it’ll be just an extension of that,” he said. In three to five years, haptic devices – which work off physical contact between the user and computer – could boost the experience further: “Companies are really working on various gloves or bodysuits and devices so that it can replicate touch and feel.”

Such technologies, said Ahmed, could be a boon to healthcare. But, he added, the role of patients in agreeing to take part should not be forgotten. “Ultimately, it is about the operation, about [the patient], about his cancer care and that has to be the priority for everybody.

“The fact that patients have agreed to do this before – with the Google Glass – and again, it is quite reassuring and quite humbling.”