Councils should be given the chance to resume their "historic role" as house builders to ease an affordable housing crisis, their lobby group says.
The need was even greater following the economic uncertainty caused by the UK's vote to leave the EU, the Local Government Association (LGA) said.
It estimated that at least four million people of working age in England would need affordable housing by 2024.
First-time buyer numbers in England have fallen by a third in a decade.
The fall in home ownership among the young and rising rental costs has led to some calls for councils to step in to increase the supply of homes with a new building programme.
A spokesman for the Department for Communities and Local Government said: "Local authorities have £2.5bn in housing reserves and borrowing headroom of £3.4bn, so they should be able to build the homes their communities need.
"Furthermore, we have set out the most ambitious vision for housing in a generation, including investing £8bn to build 400,000 more affordable homes."
Between the late 1940s and late 1950s councils built more homes than the private sector. Local authorities were building 100,000 homes a year up to the late 1970s, but the election of Margaret Thatcher's Conservatives in 1979 led to a fall in housebuilding by local authorities.
In the year to the end of June, local authorities built 1,500 homes in England out of a total of 131,370 - that is just over 1%.
"Bold new action is needed to solve our housing crisis following the vote to leave the European Union. A renaissance in house building by councils must be at the heart of this," said Peter Box, LGA housing spokesman.
"The private sector clearly has an important role to play but the reality is that it cannot build the homes we need on its own, and will likely be further restricted by uncertainties in the months and years ahead."
The LGA is calling for the government to allow councils to:
Borrow to invest in housing
Keep all the money received from the sale of homes through the Right to Buy scheme
To use this and other funding to build through spin-off housing companies
"If we are to stand any chance of solving our housing crisis, councils must be able to replace sold homes and reinvest in building more of the genuine affordable homes our communities desperately need now more than ever," Mr Box said.
A separate report from the Centre for Economics and Business Research suggests that "tremors" from the vote to leave the EU will not prevent the average UK home costing about £40,000 more in five years' time. This would push up the average UK house price from £194,000 in 2016 to £234,000 in 2021, it predicted.