The Imperial College London team hope to overcome the dangerous and common problem of leaving bits of the tumour in a patient, which can then regrow.
Early results, in the journal Science Translational Medicine, showed the "iKnife" could accurately identify cancerous tissue on the spot.
It is now being tested in clinical trials to see if it saves lives.
To avoid leaving cancerous tissue behind, surgeons also remove surrounding tissue.
They can even send samples off for testing while the patient is still in theatre, but this takes time.
Yet one in five patients who have a breast lump removed still need a second operation to clear their tumour. For lung cancer the figure is about one in 10.
The team at Imperial College London modified a surgical knife that uses heat to cut through tissue.
It is already used in hospitals around the world, but the surgeons can now analyse the smoke given off when the hot blade burns through tissue.
The smoke is sucked into a hi-tech "nose" called a mass spectrometer. It detects the subtle differences between the smoke of cancerous and healthy tissue.
This information is available to the surgeon within seconds.
Tests on 91 patients showed that the knife could accurately tell what type of tissue it was cutting and if it was cancerous.
Dr Zoltan Takats, who invented the system at Imperial, said: "These results provide compelling evidence that the iKnife can be applied in a wide range of cancer surgery procedures.
"It provides a result almost instantly, allowing surgeons to carry out procedures with a level of accuracy that hasn't been possible before.
"We believe it has the potential to reduce tumour recurrence rates and enable more patients to survive."
Trials are now taking place at three hospitals in London - St Mary's, Hammersmith and Charing Cross.
Prof Jeremy Nicholson, head of the department of surgery and cancer at Imperial College London, said: "This is part of what we call precision medicine, we're trying to change the world by very aggressively translating scientific discovery in to the NHS."
Surgeon Dr Emma King, of Cancer Research UK, said: "The iKnife is an exciting development to guide cancer surgeons during operations.
"If its usefulness is supported in further clinical trials, it could potentially reduce the time spent in theatre for many patients."

It took the Tapolca seat with a majority of just under 300 votes.
Jobbik looks set to present a major challenge to the governing Fidesz party in parliamentary elections due in three years' time.
Correspondents say the party has softened its rhetoric in recent years and has gained support from mainstream voters.
Lajos Rig's margin of victory was narrow, but his win was symbolically enormous, says the BBC's Nick Thorpe in Budapest.
"The mood in Hungary is for a change of government and with Jobbik Hungary finally has a force to change the government,'' party leader Gabor Vona told supporters in Tapolca.
It re-enforces their reputation as the most successful nationalist party in Europe and is another setback for Hungary's Prime Minister Viktor Orban.
"There are times when the ball hits the post," said Mr Orban, a keen footballer, on his Facebook page as he admitted defeat.
Fidesz easily won the last national and European elections in 2014, when Jobbik took around 20% of the vote.
But since then the governing party has seen its reputation falter, with allegations of corruption, and opposition to measures like a planned internet tax and limits on Sunday trading.
Fidesz lost its two-thirds parliamentary majority in February after another by-election defeat.

The students began "feeling unwell" at Llanwern High School, in Newport, at about 11.40 BST on Thursday.
One 14-year-old girl, five girls aged 13, two girls aged 12 and an 11-year-old boy were taken to Royal Gwent Hospital.
Laboratory tests are under way to determine what the substance was.
Police said not all the pupils required hospital treatment but were taken as a precaution. Eight of the nine have now been discharged.
Gwent Police said inquiries were ongoing to find out the exact circumstances of what happened and where the substance was obtained.
The Welsh Ambulance Service sent three rapid response cars, three emergency ambulances and an urgent care service vehicle to the school.
A council spokesman said on Thursday: "Our immediate priority is the well-being of the pupils affected.
"The school will be carrying out an investigation and will be working with the police into how this incident happened."
An "intelligent" knife that can sniff out tumours to improve cancer surgery has been developed by scientists.
[NEXT_CONCEPT]
The far-right Jobbik party in Hungary has won its first ever individual constituency seat in parliament.
[NEXT_CONCEPT]
A 14-year-old girl remains in hospital after nine pupils were taken ill at school after reportedly taking an unknown substance.