Why are we so stiff?

The housebuilding industry is investing a lot of hope in Kate Barker's review of housing supply. It is hoping that she will identify planning as a major constraint on the industry, particularly with respect to its ability to respond to changes in demand.

The industry may have its wish, but it is likely to be a pyrrhic victory. Getting the Office of the Deputy Prime Minister to change planning policies and systems (which are the cause of most of the problems) will be nigh-on impossible while it is busy pushing the planning bill through parliament and rolling out ever more interventionist policies on design, product mix and planning gain.

Barker, it seems pretty clear, is going to dismiss claims that housebuilders' land-hoarding is a cause of undersupply. Her conclusion will have been helped by the willingness of a number of larger builders to share commercially sensitive information with her and describe precisely how a landbank operates as a production line for obtaining planning permission and processing all the other agreements needed to start construction. Barker also seems to understand that "nimby-driven" planning is a major problem.

But there is another reason for the industry's lack of responsiveness to demand that seems to have been too little discussed. It is the industry's structure, and particularly the effects of – and reasons for – the present wave of takeovers. Ultimately, what the City wants from publicly owned companies is probably the exact opposite of what is needed to increase housing output and improve demand responsiveness. This conflict may thwart any actions that Barker and the Treasury take to increase output.

The housebuilding industry's output from the middle 1950s to the 1990s was highly elastic.

Just two examples: between 1968 and 1975, output fell by half and between 1981 and 1988 it almost doubled. What has been different since 1990 is that, despite the strong recovery after 1996 and the boom conditions in 2001/02, housing output has virtually flatlined at a historically mediocre level – about 145,000 to 150,000 homes a year.

Housebuilders are undoubtedly right to point to the planning system and planning policies – which have been in a sort of Trotskyite permanent revolution since 1991 – as a major source of delay and disruption in bringing land forward. The effects of introducing PPG3 in 2001, without any transition, have been shown to have been particularly disruptive.

However, in parallel with that, we have seen an enormous reduction in the number of players in the industry. Twenty years ago there were between 20,000 and 30,000 "registered housebuilders", only half of whom built anything in any one year, and often it was not the same half from one year to another.

In the current climate, where houses seem to be viewed by ministers and CABE as some kind of public art rather than something to live in, the contribution of smaller and less formal industry players is no longer wanted

Then, up to 10,000 "housebuilders", building fewer than 10 houses a year, contributed about 14-15% of the industry's output, year in, year out. Now only 4000 builders of this size are active out of a total register of only 15,000 and account for just 8% of output, while the number of firms building 31-100 houses has halved and their percentage contribution has amost halved.

In the middle range, housebuilders contributing between 100 and 500 houses have halved in number, their output falling from 20% to 14% of the total. The top 20 – all public companies – have meanwhile raised their share of a smaller output by 20%, to about 60%, during the past decade.

It was the flexibility of these smaller and less formal industry players – privately owned and able to increase output by ferreting out smaller sites or switching to contracting or into other businesses altogether when demand fell – which provided the elasticity in previous generations Many of these were true speculators – people would probably call them "cowboys" today – with no long-term commitment to the housebuilding industry. They simply invested their capital in whatever happened to be booming at the time. However, they built houses and helped to meet demand. But in the current climate, where houses seem to be viewed by ministers and CABE as some kind of public art rather than something to live in, their contribution is no longer wanted.

In contrast, neither the major public companies nor the City want wild surges in output. The majors have been the backbone of the industry's output but they have never provided much elasticity; that has has always come from the smaller firms.

Nor, indeed, does it make much commercial sense for them to do so.

But the combination of planning complexity, increasing regulation and higher capital requirements has caused many smaller operators to give up or to cash in their assets, while the City is promoting consolidation as the key to reducing risk and improving price–earnings ratios for the major public companies.

Combined with inflexible planning, these twin blights of over-regulation and City-driven of public companies have produced an industry which no longer has the inherent flexibility to respond rapidly to market demand that it had 20 years ago.