In his first Mansion House speech, Mr Osborne said he would abolish the current system of financial regulation.
The Financial Services Authority (FSA) will "cease to exist in its current form", he told the City of London.
But he also revealed that Hector Sants, the chief executive of the FSA, would stay on to oversee the transition.
The chancellor added that Financial Secretary Mark Hoban would set out further details to parliament on Thursday.
By Norman SmithChief political correspondent, BBC Radio 4
For those in the City who hoped it might be back to business as usual under a Conservative-led government George Osborne's Mansion House speech was a wake up call. Apart from the scrapping of the FSA - and the handing over of its regulatory duties - to an increasingly all powerful Bank of England,  Mr Osborne also confirmed there would be a bank levy and further action on pay and bonuses. But most significantly he announced a new banking commission to overhaul the City. It will examine breaking up the major banks; the competitiveness of the City;  whether power had become too concentrated among some leading city institutions; and whether there should be  restrictions on bank activities. In Mr Osborne's words it will be  "a new settlement between our banks and the rest of society". Mr Osborne may have donned the traditional black tie outfit - for his Mansion House speech  - but it is clear he believes many other City traditions are going to have to change.
Earlier, Mr Osborne had told the House of Commons that Sir John Vickers, former head of the Office of Fair Trading, would chair a commission to look into the potential break-up of the UK's biggest banks.
The independent commission will take at least a year to review whether casino-style investment banks should be split from deposit-taking institutions on the High Street.
In his first keynote address at the Lord Mayor's annual dinner to the City, Mr Osborne said the new coalition government was proposing a new system of regulation that "learns the lessons of the greatest banking crisis in our lifetime".
Mr Osborne paid tribute to Alistair Darling, his predecessor as chancellor, but went on to criticise the current tripartite system of regulation, which divided responsibility between the Bank of England, the FSA and the Treasury.
"No one was controlling levels of debt, and when the crunch came no one knew who was in charge," he said.
As expected, he confirmed that the FSA would be broken up and the part that monitors financial institutions would continue as a "new prudential regulator" but would operate as a full subsidiary of the Bank of England.
The parts that are supposed to protect consumers and crack down on crime will be injected respectively into a new Consumer Protection Agency and an Economic Crime Agency.
He also said the government would create a powerful new Financial Policy Committee at the Bank of England.
Mr Osborne said that the process of reforming the regulatory system would be completed by 2012.
He also revealed that he had asked Hector Sants, the chief executive of the FSA who had been due to leave this summer, to stay on to oversee the transition.
Mr Sants will become the first new deputy governor of the Bank and chief executive of the new "prudential regulator" charged with regulating banks and other financial institutions.
Andrew Bailey, currently chief cashier at the Bank of England, will become his deputy in the new regulator.
The chancellor also confirmed that the government would introduce a bank levy and "demand further restraint on pay and bonuses".
Gordon Brown, made chancellor when the Labour Party won the 1997 general election, created the FSA following criticism that the Bank of England had failed to sufficiently regulate the UK's financial system.
But in recent years critics have said the problem with the system was that it was not clear who would be in charge in a crisis and the tripartite financial authorities needed to communicate better with each other.
The FSA, in particular, also came in for criticism for not doing enough to prevent or limit the crisis in the financial markets.
The governor of the Bank of England, Mervyn King, welcomed the new powers given to the Bank by the new government.
"Monetary stability and financial stability are two sides of the same coin. During the crisis the former was threatened by the failure to secure the latter," Mr King said in his speech, which followed the chancellor's.
Chancellor George Osborne has confirmed that he will give the Bank of England the key role in regulating the UK financial sector.