Even incurable optimists like myself would have to concede that Shaun Bailey, the Conservative candidate for Mayor of London, is a long shot. The polling is dire. The most recent survey I saw showed Sadiq Khan, the Labour incumbent, on 53 per cent with Bailey on 28 per cent. The candidates for the Lib Dems and Green Party each had seven per cent. If Khan really does have over 50 per cent of the vote then the question of counting second preferences does not arise. But the polling for that was published all the same. It had Khan on 66 per cent to Bailey on 34 per cent.  So Khan’s lead on first preferences of 25 per cent was extended further to 32 per cent. We have had plenty of magnificent electoral upsets over the years. Polls are just “snapshots”, as our proprietor is fond of reminding us. But if Bailey was to win, I can’t think of a precedent for an upset on such a scale. It would be astonishing to confound the odds implied by such a poll within a few weeks of an election.

There is also the risk that really discouraging polling becomes self-fufiling for the electoral outcome. Why donate money for a campaign if the cause is lost? Party activists become less motivated to knock on doors. News of disgruntlement appears in the media causing morale to fall in a downward spiral.

How have we got to this point?

Five years ago, Khan was certainly the favourite. But there was a sense that the race was competitive. Khan was ahead of Zac Goldsmith, the Conservative candidate, by nine points on the first round. Khan then extended his lead on the second round ending up with a victory margin of just under 14 per cent. As I kept noting last week, previewing different categories of election, polling nationally shows Labour in a weaker position than in 2016. So why should London look like such a doddle?

The capital is not overcome with Khan mania. His failures and broken promises on housing, crime, transport, and the environment, have been thoroughly documented on this site. The Mayor’s petulance in resisting scrutiny has even exasperated some of his Labour colleagues and natural allies in the media. We have witnessed a preference for photo-opportunities over serious decision-making. The Council Tax precept has risen to fund extra staff at City Hall. There has been virtue-signalling, gimmickry, and ill-judged appointments to bolster his “woke” credentials. Most tiresome of all, has been that whining, repetitive, grievance-mongering. When challenged on any deficiency, the response is that, as merely the Mayor of London, he is powerless and thus blameless. It is all the fault of central Government. The Mayor is emphatic that anything that goes wrong is nothing to do with him:

“He’s the bafflement of Scotland Yard, the Flying Squad’s despair: For when they reach the scene of crime Macavity’s not there!”

Say what you like about Ken Livingstone, there was a boldness about pushing through policies that made a difference. You might feel the Congestion Charge has proved ineffective at easing traffic jams. Or that spending on the Olympics got out of control. Or that the tower blocks he approved made London more ugly. But Livingstone did not just dive for cover and claim he was unable to do anything.

So if Khan isn’t doing such a great job, should the Conservatives make Bailey the scapegoat? Is it an issue of him being an ineffective challenger? If only it were that simple. Bailey has good credentials and experience, natural charisma, and authentically Conservative instincts. His family is of Jamaican origin. He grew up in a council house with a single mother. Then as a youth worker he set up a charity to offer an alternative to gang violence. But he also has the more establishment credentials of having worked as a Downing Street policy advisor and serving as a member of the London Assembly. I would like him to be more outspoken and independently-minded rather than staying “on message.” Though in an era where comments are so easily misrepresented to allow synthetic indignation, such caution is understandable. There was a BBC TV debate last week between Khan and Bailey which Bailey won. When Khan stood five years ago he sounded energetic – by contrast with Goldsmith who came across as rather languid. Khan has retained a tone of arrogance and gracelessness while adding tiredness and complacency.

The EU referendum has contributed to Conservative woes in London. There are many who voted Remain, who had previously voted Conservative, who are very resistant to the idea of voting Conservative again. You can give them all the comparative vaccination statistics you like. There is an emotional antipathy there which is hard to overcome – as I discovered to my cost while seeking re-election to the Ravenscourt Park Ward on Hammersmith and Fulham Council in 2018. We may have “got Brexit done”. But for some of the unreconstructed Remainers still prevalent in the capital, it will take a while before the indignation eases.

But deeper still is the demographic change. There is that “rootless cosmopolitan” aspect to London – to borrow Stalin’s anti-semitic slur. For many of us, the vitality of living in a global city has great attraction. But when you go canvassing, it is a bit easier to find Tory voters among those married couples in terraced streets who own their own home, have lived there for 30 years, and are rather settled where they are. Ethnicity and income are factors in voting, but more important is tenure – which also means age. That married couple might sell up and move to Wiltshire. A buy-to-let firm snap it up and then rent it out to young people who might be in well paid jobs, yet still embrace the fashionable socialist causes in the “culture wars” – Extinction Rebellion, Black Lives Matter, transgenderism, and so on. These are issues where Conservatives are usually too nervous to robustly engage and thus lose by default. Then when these ambitious youngsters decide to settle down – get a better job, start a family, put a foot on the “housing ladder” – often they find that doing so is incompatible with staying in London. That is a terrible indictment of London. It also drives out Conservatives, or those most likely to become Conservative. Starting a family has an impact on political outlook. Those who worry about the safety of their children take a tougher stance on fighting crime. The pressure of  household budget means less sense of selfishness in favouring lower tax.

The good news is that the Conservative decline in London is not inevitable. If a genuine housing market was to operate, then supply would increase and prices would fall. We do not need ugly tower blocks to achieve this. These only prompt more Londoners to flee to Wiltshire to start their families. On the contrary, we could see higher density with attractive new terraced streets and mansion squares. There have been some welcome indications that planning policy will be reformed to move in this direction. Unfortunately, any such transformation will not come soon enough to secure Bailey’s election.