Edward Boyd is Deputy Policy Director at the Centre for Social Justice.

Criminal justice matters for the poorest in society. The Centre for Social Justice (CSJ) has travelled over 50,000 miles and interviewed thousands of people living in Britain’s most deprived communities over the past two years. When the criminal justice system fails to rehabilitate offenders, it is these neighbourhoods that pay the highest price.

Chris Grayling took over from Ken Clarke in 2012, and immediately injected energy and pace to the Ministry of Justice. He has instigated a number of wide-ranging ranging reforms to legal aid, the court system and the probation service, to name just a few. While there are important areas that the Ministry of Justice still need to tackle with urgency – none more so than the concerning spike in prison suicides – the reform of the probation service is encouraging and holds significant promise. It has been met with predictable hostility from the unions who have called it an “untried and untested ideological experiment”. Yet scratch beneath the surface and it is not hard to see that this reform is not just welcome, but is desperately needed.

It is made increasingly important by a recent change in the nature of crime. Put simply, while fewer people are committing crime we are struggling to rehabilitate a hardcore group of offenders who keep coming back to prison time and time again.

The statistics tell the story: there has been a 30 per cent rise over the last decade in the number of people going to prison for serious crimes who have already committed at least 15 previous offences. Offenders are being given sentences that are not changing their criminal behaviour.

This is primarily the probation service’s responsibility, and it is right to be ambitious in improving it. Any Government concerned with protecting communities should not tolerate the fact that almost six in ten of those given short sentences are convicted of further crimes within a year of being released. We need a more effective approach to turning round the lives of the most dangerous and broken people in our society.

Chris Grayling’s “Transforming Rehabilitation” reform seeks to address this issue head-on. It will contract out the responsibility of managing low and medium-risk offenders released from prison and on community sentences. Those delivering the contracts will be made up of voluntary and private sector organisations, whose pay will be partly dependent on their success at reducing reoffending.

We should ultimately judge this reform on whether it reduces reoffending. Whilst we won’t know this for several years, there are a number of reasons to be optimistic.

First – and most importantly – short sentenced prisoners will get the support they need when leaving prison. Those serving sentences of less than 12 months currently leave prison with just £46 in their pocket. No further support is provided. This is a scandal that costs up to £10 billion and harms countless lives every year. The previous Government tried to fix this problem with “Custody Plus”, but it was too expensive and was never implemented as a result.

Second, the reform will give the best voluntary organisations a far more prominent role. The CSJ has long witnessed the incredible power of such organisations to transform lives. Over 300 of them feed into our work and we would be nothing without them, as Peter Hoskin appreciated on his visit to our annual charity awards last week.

Their effectiveness stems from an ability to work relationally, rather than bureaucratically – proving to an offender that they care about them and their rehabilitation. They are often experts at making offenders believe they can change their ways and receive a second chance, rather than feel like they are part of a box-ticking exercise. This is a remarkably effective approach.

The probation service has been urged for years to involve them more. The last Government set a target of handing out a tenth of the budget to the voluntary and private sectors. This never happened. At the beginning of the parliament less than two per cent of their budget was passed onto charities. The reform will change this as three-quarters of the supply chain are either charities or mutuals.

Third, it will tackle needless bureaucracy. Incredibly, probation officers only spend a quarter of their time engaging with offenders. The rest of the time is spent filling out forms or at meetings. The reform will replace pointless bureaucracy with the space for innovation by incentivising providers on outcomes, rather than holding them accountable solely for inputs.

Finally, the providers pay will partly depend on whether they reduce reoffending, not how they do it. This will promote innovations and new ways of working – experimentation that is desperately needed if we are to crack the difficult problem of persistent reoffending.

This should put a stop to the self-serving behaviour that has developed over years of judging performance on inputs. The story of one charity – who train volunteers to mentor ex-offenders – is instructive. They offered a probation trust 30 volunteers to mentor ex-offenders free of charge. They were turned down. The probation trust told them it was too risky to take on volunteers and there was a fear that the success of volunteers might undermine probation officers’ jobs.

There are still a number of factors that could easily de-rail this reform. For instance, new providers will have to form relationships with partner organisations quickly, deal with offenders who become high-risk effectively, and learn to do far more with a lot less money than their predecessors. They will also need to do this in partnership with a prison service that is struggling to play its part in rehabilitating offenders. None of this will be easy. Yet the scale of the problem requires a radical response, and this is what is being delivered. The ambition should not be faulted. Crime-ridden communities across the UK will be hoping that the reforms live up to their promise.