During the 1980s, across much of England and Wales, there were fewer new houses built than there was growth in the number of households. So by the time of the 1991 Census in some parts of the country there were only slightly more dwellings than households, as we can see in the following figure.
It was feared that even this might under-state the problem, since the 1991 Census was widely believed to materially under-estimate the population, with large numbers of people refusing to register for the Census or the electoral register in an attempt to avoid the Poll Tax.
During the 1990s this problem seemed to be getting worse. Inter-censal population and housing estimates suggested that in London and the South East there had come to be fewer dwellings than households – households were doubling up in the same dwellings. This culminated in the 2000 Mid-Year-Estimates (MYE) of the Department for Transport, Local Government and the Regions (DTLR), as here:
Thus was born the notion that there was a "housing shortage" in London and the South East of England. As house prices began to spike upwards in the early 2000s, that was widely attributed to the housing shortage. It was claimed that house prices were rising because people were desperately out-bidding each other simply in order to have somewhere to live. The government established the Barker Review of Housing Supply to investigate how housing supply could be increased, and the Communities Plan for increased housebuilding.
But the whole concept of there being a "housing shortage" in London and the South East was false. It was based on estimation errors. There were three major ones in the inter-censal estimates:
a) It was assumed that the 1991 Census markedly under-estimated the population because of poll tax avoiders refusing to complete the census, with estimates of the number of people “added back in” – but the number of actual such people turned out to have been much less than estimated
b) As part of the Single Market programme we stopped measuring properly who left the UK for the EU, but were still punctilious in measuring everyone who came in. So if you were an Australian who went on holiday to Italy and decided to stay, then came back to the UK to pick up your stuff and left for Italy again, at the end the inter-censal population estimates might show two Australians the UK when in fact there was no-one here.
c) In the 1990s inter-censal estimates of births assumed every new baby being registered in a given NHS administrative area was a new birth. But asylum-seekers were moved around very frequently, with the result that in some cases they moved between up to 12 NHS administrative areas within the first six months of a birth – leading to bizarre stories in the press about high birth rates amongst asylum-seekers.
The 2001 Census was the most accurate ever conducted to that point. It showed that there were 900,000 fewer people in England and Wales than the inter-censal estimates had suggested – a massive error. The margin of error for the 2001 Census was only +/-0.2%, or around 100,000 people – so much less than the difference. And what was true of England and Wales as a whole changed the regional picture dramatically, as we can see:
Instead of surpluses of dwellings over households falling across England and Wales and going negative in London and the South East, they actually increased everywhere except London where they stayed constant and comfortably positive.
Europe Economics wrote about this in 2004 on behalf of the CPRE and was fairly widely reported. However, during the 2000s, however, house prices rose very rapidly, before falling back 30% in real terms from 2007 on (and still falling). The notion of a "housing shortage" in London and the South East was so ingrained that it seemed invulnerable to the facts. Perhaps there was something aberrant about the 2001 Census and things would reverse with the 2011 data?
Well, those 2011 data have been released today and they are conclusive: there is no "housing shortage" in London and the South East of the sort identified in the 2000 Mid-Year-Estimates and other data from the late 1990s, and never was.
Does any of this prove that UK planning policy is perfect? Obviously not. Ought the UK debate about housing to have been transformed by the 2001 Census data? Surely. Will everyone stop talking about a "housing shortage" now, having done so wrongly for about 15 years? Surely not. But one day the discussion will catch up with the facts.