Rachel Wolf is a partner in Public First. She was an education and innovation adviser at Number 10 during David Cameron’s premiership, and was founding director of the New Schools Network.

“Don’t cross the home-schooling lobby” was accepted wisdom when I worked in government. In 2009, Labour attempted to introduce a home education register – with a requirement on parents to submit a plan for their children’s education that the local authority could inspect against.

They backtracked rapidly in the face of furious and well-organised opposition from parents. This became a model of how parents can make a difference, and who not to cross in government.

The points made by those parents were very Conservative: this is my family; the state has no right to interfere; why are you assuming I am doing something wrong with no evidence?

I am deeply sympathetic to this view. One of the many things I was unprepared for when I had children was the visceral panic and fear of the state deciding to get involved – right from that first Health Visitor interview two days after giving birth (where they try to subtly ask you if your partner is beating you up as they mark on a piece of paper if social services should get involved in your family).

I’m not alone. Two doctor friends of mine told me not to take my children to hospital immediately if they had an injury but to call them instead – just in case it triggered some kind of investigation. I know it sounds absurd, but the occasional story of children being wrongfully taken from their parents is beyond fear-inducing. For parents who home educate – many of whom will have had bad experiences at their children’s initial school, and many of whom are likely, philosophically, to disagree with the state’s approach to education – that fear will be much greater.

And strong parental interest in their children’s education is exactly what, in other areas, government is desperate to have more of. Whether it’s healthy eating, or reading to them in the evenings, or just turning up to parents’ evenings, government is constantly trying to persuade parents to do more. It seems paradoxical to somehow take action against this group of parents who are making active and sometimes unfashionable decisions to support their children – often at great financial and opportunity cost to their own lives.

So why have I come – reluctantly to accept that the Government has to track home educated pupils (as it proposed last week)? Because there are some frightening things happening with a growing number of children that currently we know terrifyingly little about.

Best estimates are that the number being home schooled has almost doubled from 34,000 in 2015 to 60,000 today. We don’t know if this is an accurate number, because no one tracks these children. It’s the best estimate of local authorities – and is quite likely to be too low.

We also don’t know why it has increased – again, because we don’t track. Anecdotally, there are half a dozen reasons. Two common ones are: a) because some schools are chucking out pupils (“off rolling”) before exam time to boost their results; and b) a growing number of students who are ‘home schooled’ aren’t being educated at home – they’re going to an untrackable number of “unregistered” schools (usually religious).

Here’s an example of one of the schools in an Ofsted letter from the previous Chief Inspector, Sir Michael Wilshaw:

‘At the most recent visit, inspectors were intentionally obstructed from entering the premises for an hour. During this time, inspectors observed a number of female students to be on the premises. When inspectors finally gained entry, the girls had left the building by an alternative exit. The staff on site informed inspectors that these girls were attending a local library and that they would return later in the day. They did not return and the staff were unable to account for their whereabouts for the duration of the inspection visit. Inspectors also found squalid conditions, including three single mattresses covered in filthy sheets in one room and no running water in the toilet areas, and clear evidence of segregation, with separate classrooms for boys and girls.’

This leaves aside the radicalising material that has been found in some unregistered educational settings in recent years.

This, then, is the core tension. How do we weigh up our responsibility to very vulnerable children – and to protecting others from their later actions – against the rights of most families? If some children are being radicalised, or forced into a life because they are women that we would view as unacceptable in modern Britain, or simply being given a rotten education, does this give us the right to interfere with everyone? Particularly knowing that we are going to be causing at least some stress and bureaucracy to families that are doing a very good job?

There is no way round that trade off. It’s one where, often, I think the government comes down on the wrong side – as I wrote in my last column, I don’t think the proposed pornography age-checking service is really justified.

On this, though, I think we have to accept that we live in a new age – one where it is too dangerous for the children concerned (and potentially to us) to live in a vacuum of information. I am also deeply concerned about the education girls might be receiving (or, which is another thing our lack of visibility into children who are withdrawn from school stops us seeing, the number of underage marriages they might be pushed into). Yes, I hate the idea of the state interfering with my child – but I also hate the idea of my daughter growing up in a country where all women aren’t getting access to a proper education and opportunities.

It is reasonable for the Government – even though it will be annoying to many parents – to expect to know where these students are, what they’re doing, and therefore be able to act if something terrible is going on.

But there are going to be negative consequences on parents who are not only within their rights but who are doing the right thing by their children. We have to recognise this, and apply the highest possible bar for interfering in what those parents are doing. If it’s not destroying their children or likely to seriously jeopardise our safety, we need to leave them alone.