The Local Government Association is asking that councils in England be freed from restrictions on the ability to borrow to fund new home building.
It is also calling for powers to replace every council home that is sold off, as quickly as possible.
The government said it was delivering on its commitment to replace all properties sold on a one-for-one basis.
But in a submission to the Treasury, the LGA said the scale of the housing crisis demanded that action to fix the lack of affordable housing was taken immediately.
"The government's recent announcement of an additional Â£5bn investment in the form of loans to private sector organisations and for building homes on surplus public land is a welcome step," the LGA said.
"However, councils are also well placed to bridge the gap between housing need and future building levels and are keen to see - and support - delivery at speed."
The LGA wants to hold talks with ministers over whether councils' housing revenue accounts  - the accounts covering rents and assets - could be removed from the public sector debt book so as to enable borrowing against these assets.
"This would allow local government to rapidly contribute new homes in the market, meet a wide range of local needs and generate significant medium and long term returns," it said.
Councils also want powers to replace council homes that are sold off under Right to Buy arrangements.
Currently, the government only allows local authorities to keep 30% of the cost for replacement council homes.
This means councils cannot build replacements - especially as they are unable to borrow to fund the rest.
The call comes as new figures were released on the sales of social housing.
These revealed nearly 22,000 social housing dwellings were sold from 2015-16 - 12,557 by local authorities and a further 9,435 by housing associations.
The sales amount to about 0.5% of the total stock of four million social houses in England.
There was a particular spike in the percentage of housing association homes sold to private firms.
This was up 46% and government statisticians said this was likely to be because of housing associations taking a more active approach to managing their assets.
The Department for Communities and Local Government said it was delivering on its commitment to replace all properties sold on a one-for-one basis.
"As of June 2016, there had been 7,018 starts and acquisitions.
"This delivers more than a one for one replacement on the 4,369 additional sales since the scheme was reinvigorated in 2012."

The bridge over the M20 near Maidstone in Kent was hit by a lorry carrying a digger on Saturday.
A motorcyclist suffered broken ribs and a lorry driver was treated for shock.
The M20 will be closed between junctions 1 and 4 coast-bound and junctions 4 and 2 London-bound from 20:00 BST on Friday until 06:00 Monday.
The collapsed section of the pedestrian bridge, near the village of Addington, was removed from the scene last weekend, and the road was reopened a day later.
Highways England said the remaining structure was being constantly monitored and was structurally sound and safe for traffic to pass underneath with a temporary 50mph speed limit in place.
Chief highway engineer Catherine Brookes said: "We naturally need to remove it under safe controlled conditions this weekend.
"We will use the closures to carry out as much work as possible, including barrier repairs, resurfacing and litter picking.
"We will start planning the replacement in due course."
The coast-bound M26 will also be closed from the M25 junction 5.
Diversions will be in place via the A229 and M2/A2 to join the M25 at junction 2.
Local traffic will be able to use the A20.
Access to the A21 from the anti-clockwise M25 will also be closed with a diversion via junction 4.
The M20 is the main route to the Channel Tunnel and Port of Dover.
Town hall bosses want urgent talks with the government on how they can rapidly build homes where they are needed.
[NEXT_CONCEPT]
A footbridge over a motorway which partially collapsed when it was struck by a lorry will be removed at the weekend.