The bomb, measuring 5ft in length and weighing 1000lb, has been found in Southwark by builders, the Met Police said.
The force said officers were called to a building site in Grange Walk at 09:18 GMT. Two schools have been evacuated.
A cordon and a 400m exclusion zone have been put in place as a precaution.
Traffic is being diverted and several roads are closed. The council said there was "major disruption" in the area especially around Tower Bridge and Grange Walk.
It advised people to find other routes home.
The Met said it was working with the London Fire Brigade, London Ambulance Service and Southwark Council.
Leader of Southwark Council, Councillor Peter John, said: "I can understand how worrying this is for local people, and the council is doing all we can to support them while the emergency services make the area safe."
He said there were "well-rehearsed" emergency plans in place and urged residents to go to one of the council's rest centres in Canada Water for hot meals, tea and coffee.

The isle is a mile long (16.km), depending on tides and weather, and separated from the mainland by a (330ft) 100 metre wide channel.
It was first spotted by locals as a "bump" above the water in April, according to the Island Free Press.
Photos taken by a Connecticut tourist have gone viral on Instagram, leading to a boost in curious visitors.
Photographer Chad Koczera described encountering the island during his visit to Cape Point.
"My fiancee and I were driving to the point after a storm to collect shells when we spotted an area we couldn't get to by car," he said.
He described returning for his fifth time to the park, and to visit Cape Point where he had proposed marriage to his then-girlfriend.
"I sent the drone up to check it out and noticed this beautiful island. We didn't get a chance to make it on to the island because of the strong current."
The island is littered with old shipwrecks and whale bones that poke through its sands, says local historian Danny Couch.
Experts say the shores of the Outer Banks are constantly shifting, as sandbars rise above or sink back below the water.
Park superintendent Dave Hallac has cautioned visitors against trying to swim or walk across the channel to reach the island.
Currents can swiftly carry away a person, while sharks and sting rays have been spotted in the area.
Visitors are advised to use a kayak or paddleboard to reach Hatteras Island, as it's been dubbed.
More than 1,000 homes in south London have been evacuated after an unexploded World War Two bomb was found on a building site.
[NEXT_CONCEPT]
A new island has formed off the coast of North Carolina's Cape Hatteras National Seashore.