The Rt Hon Dr Liam Fox MP is former Defence and International Trade Secretary. He was the UK’s Nominee to be Director-General of the World Trade Organization (WTO) in 2020

In the last Conservative leadership election, I did not vote for Boris Johnson. He subsequently sacked me from the Cabinet as he was perfectly entitled to do. I, therefore, cannot be accused of being a sycophant in writing that this is absolutely the wrong time for the Conservative Party to think about a leadership challenge.

Even if we reach the effective end of the pandemic, there will be an enormous task to recover at home and abroad. Global trade will continue to be disrupted for some time and demand downturn in the stronger economies could potentially become a problem for developing nations in their ability to service their debts.

Across the globe, inflation continues to cast a dark shadow with central banks too slow to react as part of their “groupthink” belief that the problem is “transient”. In Britain, a generation that has never experienced the horrors of inflation will learn that it hits the poorest in society hardest.

Politically, the challenge that inflation will bring to the public finances will make it difficult to splash out on big public spending projects, though this may be the silver lining. Abroad, we have an increasingly assertive China making belligerent noises about the South China Sea and Taiwan and, closer to home, the threat of Russian military action against Ukraine. It is a difficult and dangerous period.

This is a time for the whole government to concentrate its efforts on the substantial tasks at hand rather than engaging in a bout of navel-gazing that will lead to division and paralysis. It is not a time to be led by what opinion polls tell us.

Those who hate Johnson because of his role in the referendum campaign, or because they are unwavering in their opposition to the Conservative Party, will never be placated. We should not be swayed by their voices.

Equally, we need to understand the weariness of the public after two years of Covid-19 restrictions of one form or another. As more familiar issues return to centre stage in our politics, the public will expect progress on a wide number of fronts which is all the more reason to focus on delivery. We also need to break away from a culture that sees politics as some sort of X Factor contest where personalities become more important than the substantive issues of the day. We do not need potential candidates forming shadow campaign teams within the parliamentary party with the inevitable diversion of energy and division.

It is easy for us to forget the state of our politics when Johnson became leader of the Conservative Party. There was a clear attempt by those who had campaigned to remain in the European Union to prevent the democratic will of the electorate from being carried out. It was, in effect, an attempted political coup against the British people.

Theresa May profoundly believed that it was a matter of honour that she should deliver the verdict of the Referendum but, after her parliamentary majority was lost in 2017, the forces of the opposition parties, a number of Conservative MPs, and the appalling – and unconstitutional – behaviour of the then Speaker, John Bercow, combined to put the whole Brexit process at risk.

The election of 2019, under Johnson’s leadership, produced the biggest Conservative majority since Margaret Thatcher’s win in 1987 and ensured that Brexit was completed. Just how satisfactory the Brexit agreement will be in the long term remains an active topic of debate, especially in relation to Northern Ireland.

At the time, the most frequent complaint was that Brexit had come to dominate the political landscape so much that all other issues had been pushed to the sidelines. People would ask, “Will we ever get to talk about anything other than Brexit?”. Careful, as they say, what you wish for.

No one could have foreseen how, less than two months after the Conservative election victory of 2019, the world would be gripped by the Covid pandemic. Never has a single issue altered our political discourse as much or consumed so much government bandwidth.

Despite inevitable mistakes being made (which every rational person understands), if the Government had not had both the freedom and wisdom to order large quantities of vaccine in the early stages of the pandemic, Britain would not have been able to produce the world-leading vaccine campaign that we did.

At the same time, the Prime Minister led the G7 and maintained a strong interest in the whole issue of climate change at a time when many others regarded it as an unnecessary sideshow.

This is not remotely to suggest that all is well in the Johnson premiership. For many Conservatives, including myself, the current government smacks too much of “big tax, big spend, big state” more reminiscent of Edward Heath than Margaret Thatcher. The question of the Northern Ireland border remains a thorny subject, and for many a continued border within the United Kingdom is incompatible with the entire ethos of a Conservative and Unionist Party.

The current investigation into whether Covid rules were broken has opened a “one rule for one and another rule for others” narrative that will be hard to dispel. Perhaps more importantly, it has exposed what many of us have believed for some time to be a chaotic internal management system. Johnson has many strengths. Campaigning is one of them, administration is not.

It often takes time for Prime Ministers to understand that the mechanics and organisation of government matter. Sir Tony Blair describes how it took his entire first term for him to realise that when he thought he was pulling levers, he was, in fact, pushing string. It is essential that both sound administration and good political instinct are combined in Number 10 if the Prime Minister is to have any longevity.

One of the most precious commodities for any Prime Minister is their time. It is important that the Prime Minister builds the right team around him, with someone with the authority to make the right calls on what needs to go into his box and to decide which ministerial papers can be signed off on his behalf to prevent a logjam of decision-making in Whitehall. As a former Secretary of State, I know that finding such a person isn’t easy. They require a range of qualities: loyalty, trust, reliability, and especially for our current climate an incredible amount of political experience and a well-practiced political antenna.

He also needs to take an interest in the reshaping of Whitehall.

So, while the current opinion polls are much less flattering than in recent times, we need to remember two things. First, we are 11 years into a period of Conservative government (or at least Conservative led) and it would be more remarkable if we were not having a dip in the polls.

Second, governments have recovered from much worse positions than this, as those who remember the more difficult times of the 1980s will attest. Leadership changes can bring about short-term improvements in political fortunes, but the internal wounds can leave long lasting scars, as the political assassination of Thatcher proved.

Given that the pandemic has become the central and defining issue of Johnson’s government so far, there is some merit in the argument that we will only know the Government’s real agenda in the coming months and that we should defer judgement. This is a time for unity over division, hard work over personal ambition, and putting the country before party. It is not a time for a leadership challenge.