Mark Boleat said London would remain a leading financial hub, with only a few banking jobs likely to move.
Urging speedy trade talks, he added: "We would hope that the negotiations go quickly and go well."
The triggering of Article 50 of the Lisbon Treaty last month began the two year countdown to the UK's EU exit.
Before the referendum, the financial services sector largely backed the Remain campaign, warning that quitting the bloc could spur an exodus of City jobs.
The City of London, the council that covers London's financial centre, itself backed EU membership.
Since then, banks including Goldman Sachs, HSBC and UBS have said they will move some jobs out of London as a result of Brexit.
The historic insurance market Lloyd's of London has announced plans to open a Brussels subsidiary in early 2019.
But in an interview with the AFP news agency, Mr Boleat said that while a few people would be moved, "no one is going to say 'we are closing down in London'".
"Banks have had contingency plans since before the Brexit referendum in some cases," he said.
"They have known there is a possibility that Britain will not be in the single market - that has now been confirmed."
He said banks would need to "restructure what they are doing - perhaps cease doing a few things, in some cases build up business elsewhere in the European Union. They are getting on and doing that."
Mr Boleat also said it was important the UK reached an early agreement over the terms of its exit from the bloc and on the rights of EU nationals.
That would minimise disruption to "business and consumers in this country - and in Europe," he said.
On Friday, the Bank of England governor warned of sector-wide consequences if the UK left without a trade deal.
Mark Carney said the "vast majority" of City firms already had contingency plans in place.
However, he said some financial firms still needed to prepare in case of a "more extreme" outcome.
The Bank has written to the UK's biggest financial firms, urging them to plan for "all eventualities" from the UK leaving the bloc.

Labour's Barry Sheerman said the change "shrinks childhood" by making people adults at 16.
He spoke as MPs debated extending the franchise during the EU referendum, a move they then rejected in a vote.
Shadow Europe minister Pat McFadden said he respected Mr Sheerman but could see no link between extending voting rights and sexual abuse.
Huddersfield MP Mr Sheerman said: "Up and down this country we've had vulnerability to sexual predators and ghastly things happening right through to 18 and this move to have adults at 16 will make a lot of young men and women more vulnerable to sexual predation than that happens at the moment."
After Mr McFadden rejected this argument, Mr Sheerman added "what those of us - the small minority of us on these benches I have to say - worry about is" that if people are treated as an adult at 16 "that shrinks childhood".
Labour and the SNP support allowing 16 and 17-year-olds to vote in UK elections, but the government is against the policy.
Setting out his party's position, Mr McFadden said that aside from the wider debate about the voting age, there was a "very strong reason" to lower it for the EU referendum because young people's future was "directly at stake".
If the UK votes to leave the EU, their ability to live and study in Europe would be affected, he said.
But Conservative Cabinet Office Minister John Penrose said this was not the time to consider such a change, saying the issue should be "soberly and properly debated" on another occasion.
The change was rejected in a Commons vote by 310 to 265.
The bill, which will put into law the in/out referendum promised by 2017, is currently making its way through Parliament.
Also during the committee stage debate, an SNP motion to allow EU nationals resident in the UK to vote in the referendum was rejected by 514 votes to 71.
The outlook for the UK financial sector has improved since Brexit was triggered, the policy chief for the City of London Corporation has said.
[NEXT_CONCEPT]
Allowing 16-year-olds to vote will make them more vulnerable to sexual abuse, an MP has said.