Officers spoke to him on Saturday, but he was found not to be breaking any law and was not arrested.
The Met's chief, Sir Bernard Hogan-Howe said: "It may not have been the flag. But I would like to get to the bottom of it and we will investigate it."
He said on average one person a day was arrested for terror-related activity.
Pictures of the alleged incident emerged over the weekend.
Sir Bernard told the BBC's Victoria Derbyshire programme: "The law says clearly that if you're a member of a proscribed organisation which IS is, or appear to be, then of course the police can take action.
"Just having a flag itself, you've got to make a few more inquiries."
He said it was vital it was established whether the flag did represent IS or not.
It was also put to Scotland Yard's chief that a child had been seen waving a miniature version of the flag, which he conceded "would be a worrying thing" if accurate.
He said: "I don't want to over-react because it may not have been the flag. But I would like to get to the bottom of it."
The force has faced a backlash on social media about its decision not to arrest or question the man further.
Umair J Malik tweeted it was "utter nonsense" that the man had not been arrested, while Malen Zell also called it a "joke".
He tweeted: "Utterly disgusted and sickened that a man was allowed to walk the streets of London draped in an #Isis flag."
Sir Bernard said the country was currently on the "severe" terror alert with only one level above it.
He said: "It can't get much higher in terms of threat, and we know that it's been aggravated by the fact that so many people have gone to Syria.
"What it does mean is there are quite a few people out there who we need to keep an eye on and make sure they don't get away with their plans."
A full interview with Sir Bernard Hogan-Howe can be watched on the Victoria Derbyshire programme.

Ali Mahmod, of Newbolt Court, Felling, was found semi-conscious on Sunderland Road at about 15:00 BST on 2 May and later died.
Gary Pitt, 22, of Falstone, Leam Lane, is due to appear at Gateshead Magistrates' Court later.
A 26-year-old man has been released with no further action, Northumbria Police said.
Scotland Yard is investigating whether a man stopped by police outside the Houses of Parliament was draped in an Islamic State (IS) flag.
[NEXT_CONCEPT]
A man has been charged with manslaughter following the death of a 42-year-old in Gateshead.