Before we go any further, a declaration: my wife works for a prison rehabilitation charity. I have no idea if they will benefit from Chris Grayling’s newly announced programme of more intensive efforts to prevent convicted offenders from returning to a life of crime – they may do, I haven’t asked – but it’s worth putting on the record.

It was an encounter through that charity that got me thinking more deeply about the conservative responsibility to promote rehabilitation. Waiting, late at night, outside a Government building to help my other half pack up after a fundraising event, I got chatting to the policeman on duty at the entrance. He asked what brought me there, so I explained the connection. “And what is it you do?” he followed up – sadly failing to ask “What’s all this then?”. I told him about ConHome, and the sometimes peculiar world of political blogging.

“That’s a weird match, though – Tories and prison rehab don’t exactly go hand in hand, do they?”

He was, of course, fundamentally wrong. Yes, it is conservative to believe in firm punishment for committing crimes – but it is also conservative to want to prevent criminals from reoffending. We want less crime, and we want lower taxes – every new offence harms the former goal, and every person who must be sent back to prison at exorbitant expense harms the latter.

Consider the following statistic: of the adults released from custody between April 2012 and March 2013, 45.1 per cent of them reoffended within one year. That’s the proven rate – meaning in all likelihood there were others who weren’t caught.

The good news is that is the lowest rate since 2002 (yes, it has been even worse than it is now). But that’s about as far as the good news goes. Tens of thousands of people are leaving the prison and falling straight back into the lives, habits and surroundings that led them to commit crime in the first place. As a result, more victims suffer, the taxpayer has to cough up yet again to put them back in the slammer and the grim life of the habitual criminal continues.

Prison works, as Michael Howard declared, but it must work in two ways. First by the simple act of keeping a would-be offender off the streets and thus away from potential victims for the duration of his or her sentence. That’s the easy part. The second function must be to prevent them offending again.

There is a flawed division in the way we in politics talk about these two goals. Prison policy must not be a choice between either punishment or rehabilitation – it must provide both. Indeed, as a conservative I would argue that punishment is a crucial element of rehabilitation.

This is why we should recognise Grayling’s push for better support – particularly for those prisoners re-entering society – for what it is. A practical measure to reduce crime. A structural reform to reduce the burden on taxpayers. And a moral mission to turn around the lives of those whose losing battles with life cause misery and pain not just for themselves but for others. An ex-prisoner who is also an ex-offender is a success story for all of us – we should seek to create many more of them.

Just as IDS transformed the Conservative approach to welfare reform by insisting that fiscal good sense went hand in hand with social responsibility, so we must insist that when it comes to prisoner rehabilitation, compassion and crime fighting are inextricably linked.