Tim Brighouse is a former Commissioner for London Schools, and Chief Education Officer for Birmingham and for Oxfordshire.

If civil servants ask for an off-the record discussion about a forthcoming White Paper, I always say ‘yes’. When therefore some DFE officials wanted to talk about a book ‘About Our Schools; Improving on Previous Best’ I had just written with Mick Waters, I didn’t hesitate. And I shall of course respect their confidence.

I could understand their interest because we had interviewed over a hundred witnesses, each for an hour by zoom over lockdown, before arriving at our conclusions of what was preventing our schools from being very much more successful. Among them were 14 secretaries of state, five schools ministers, some spads, heads of Ofsted, senior civil servants (including a couple of Permanent Secretaries) and a wide range of CEOs of multi-academy trusts, headteachers and teachers.

Our book is full of their very frank comments about the successes and failings of the schooling system .

As for the key witnesses, the ministers, we were naturally curious about their approach to the job. Did they start with a ‘to do list’ from the Prime Minister? What was the role of the special advisers? Who else influenced them? What were they most proud of? Did they regret anything? (In fact, few regretted sins of commission although quite a few things they hadn’t managed to do.)

Finally – relevant to a White Paper – what would they do now if they were still Secretary of State?

All saw spads as crucial – indeed, those who didn’t stay long and achieved little put it down partly to the failure to get them in-post quickly enough. All four secretaries of state who had the most influence emphasised spads’ vital contribution.

These were Kenneth Baker (delegated budgets to schools and National Curriculum), David Blunkett (school Improvement and Urban Education), Ed Balls (bringing education and children’s services together under the ‘Every Child Matters’ agenda) and Michael Gove (Ebac, and narrowing the curriculum to the basics).

Nor was it simply the SpAds which explained their success. They were all unusually well-prepared for the role with a ready-made agenda. In the cases of Blunkett (1992-2001) and Gove (2010-2014) there was a long period of being shadow spokesman prior to a general election. The other two, Baker (1986-1989) and Balls (2007-2010) had put their toes into schools’ policy from a different position in government.

Baker from the Business Department had pioneered computers in schools and run the Technical Vocational Education Initiative (TVEI) before arriving at DFE, and Balls from working on Sure Start and other matters with Gordon Brown at the Treasury.

The rest had less than 24 hours’ notice before taking the role, inherited a huge in-tray from their predecessor, and stayed for just a couple of years or less. No wonder their regrets were uniformly of things they wished they had more time to do in a job for which, with two exceptions, they had no previous experience apart from long-ago life as a school pupil.

The two exceptions, Gillian Shephard and Estelle Morris ,had both been teachers – in Shephard’s case she had also been an LEA Officer and adviser as well as Chair of the Norfolk Education Committee – and each showed in their interviews a deep empathy with the teacher’s role done well.

Charles Clarke, one of a select band of short-stayers who was called to greater offices of state, drew our attention to his fascinating book The Too Difficult Box, with cross-party contributions on topics (none as it turns out educational) which contributors saw as having been put aside as being too hot to handle because the short-term consequences were politically unpalatable.

With his book in mind therefore, when the DFE officials invited me to suggest what from the more than 40 recommendations in our book would be appropriate for the forthcoming White Paper, I didn’t mention A Levels even though many former ministers conceded that various previous attempts to grasp that nettle should have been taken.

Nor would it have been a good use of time to raise a wholesale change to Ofsted, which all ministers, irrespective of party, thought fine and fit for purpose – even though most of expert witnesses, including Sir Michael Wilshaw, former Head of Ofsted, thought it needed radical reform. As Wilshaw said:

“Deep down I knew they (critics) were right… about the inconsistency of judgements… the problem was (in moving away from data) you have to rely on the personal judgements of inspectors who may lack the personal experience or wisdom to come to the right conclusions.”

Instead, I thought the White Paper will want to deal with the governance of schools with its bewildering and confusing mix of school types – Aided, Voluntary Controlled, Community, Studio, Foundation, Trust, Stand-alone Academy, Multi-Academy Trust (MAT) and Free School to name a few.

Despite at first a stream but now a trickle of examples of unscrupulous and dishonest MAT behaviours, at its best the MAT model, committed to school improvement through schools working together, is a good one and is rooted in the transformation of performance in London’s schools. All it needs now is attention to their regulation and inspection and an ‘observer status’ local authority representative on each Trust Board.

My second suggestion was to encourage, on the back of the huge expansion of digital lessons during Covid, the creation of an Open School to which all pupils can belong as well as being a member of their local school. It should be independent of DFE and the Secretary of State, who between them have more powers leading to the most centrally directed schooling system outside North Korea.

In our book we suggest ways of dealing with that too, as well as how to reduce what others call the ‘English Disease’: the fact that for every child permanently excluded in Scottish schools (five in 2018/19) 1,500 are excluded from English Schools (7,894 in 2018/19). Gove called exclusion a ’necessary evil’. Well maybe, but surely not that frequently necessary? What are the Scots doing that we could learn from?

Small wonder that it’s not just our book drawing attention to the need for a new age in schooling- one of hope, ambition, and collaborative partnerships. The Times Education Commission will report in the summer, and their analysis mirrors our own. And there are many reports from CBI, the Select Committee, and others outlining the matters we need to fix, if our pupils are to be equipped to solve the formidable social environmental and economic problems we have bequeathed them.

Take 20% off ‘About Our Schools’ using code SCHOOLS20 at until 30th April 2022.