Nadhim Zahawi is a member of the Foreign Affairs Select Committee and MP for Stratford On Avon.

There are 85,895 people in prison in England and Wales and it costs us £3 billion a year. The price per place is just over £36,000. I start with these stats because we still have a deficit that is far too high and a growing public debt burden. It should be an obvious target for savings. But more importantly, it’s also a wasted opportunity.

Justice is always an emotive subject. When people commit crimes it makes us angry. If we’re the victims of those crimes, it’s desperate. When our house is burgled and we see the culprit get three years instead of the 50 that we would have seen them get for violating our home, it leaves us fuming. But that’s why we delegate justice away so it can be managed for the public good. Fewer people in prison and lower rates of reoffending achieve this better than soaring costs and ever more crime.

In cases of serious crimes, our position shouldn’t change. We have to see justice done and long sentences are deserved. However, most people in prison in the UK are not there for the heinous acts that inspire lurid headlines. Every three months, around 10,000 people are sent to prison on short sentences of less than 12 months. This means that these short sentences make a large proportion of the total prison population at any given point. Non-custodial sentences would be cheaper and do not risk offenders picking up bad habits in jail.

Our jails are also filled with people with mental health problems and drug addiction who need treatment more than prison, as well as too many people serving spells for minor financial misdemeanours. Yes, they’ve committed crimes and some justice had to be done. What I do question is whether we would have the same instinctive reaction to jail these people if we considered the £36,000 price tag that comes with it, not to mention the cost to the families of those imprisoned.

In August, I wrote in this column that the Government should put family policy at its heart because of the huge cost to society of family breakdown, but it’s just as important to note that more children are affected by the imprisonment of a parent than are affected by divorce.  Needless to say, children should not be punished for the crimes of their parents.

To make progress, we need to improve non-custodial sentences, so we can lock away fewer people. Fortunately, we now have a Secretary of State in Michael Gove determined to revolutionise the justice system, by doing exactly that and focusing on rehabilitation within prisons. It shows how radical our Party is today under David Cameron that so many old shibboleths are being cast aside. Conservatives used to crow “Prison works” and many held “lock em up and throw away the key” attitudes. A vote winner – until you have to deal with the consequences.

Prison can work, but only if jails are places where those who can’t cope in society can get the help and skills they need to manage afterwards. We send people to prison as punishment, we don’t send them there to be punished. It’s far better to make a good citizen and valuable member of a community out of someone rather than giving up and discarding them. As the Prime Minister’s Apprenticeships Advisor, I am also determined to add to this by pushing apprenticeships into prisons to help prisoners get the skills they need.

Given the choice between the kind of life law-abiding people lead and the dangers of a life of crime, few would really choose the latter. Most stumble into it or never gained the skills to cope normally. Circumstance plays an unignorable role. We need to do a lot more understanding than judging. I’m not going to advocate soft-headed determinism and say that it’s never anyone’s fault, or that we can never hold people accountable for their actions. It’s obvious that all people are judging and deciding individuals and therefore have a choice. It is impossible however, to get to the bottom of the problems afflicting society, without tackling the causes. To correct Tony Blair, we don’t even need to be too tough on crime, just its causes.

As Education Secretary, Michael Gove was the government’s greatest reformer, the clearest champion of our vision for a country where everyone has the right to a great education and opportunity. Now, as Lord Chancellor and Justice Secretary he can regain that mantle again. The man is irrepressible.

According to Nick Hardwick, the Chief Inspector of Prisons, many of our jails are “places of violence, squalor and idleness”. A problem with British jails is that prisoners are effectively abandoned once the cell door slams shut. That shouldn’t be the way. Under Gove, it will change. The Justice Secretary also has the backing of George Osborne to make this happen. That’s why we’re able to sell off the old Victorian prisons that are not fit for purpose, use that expensive inner city land for housing, and use some of that money to build modern, rehabilitation focused prisons that serve our aims properly.

Under Gove’s proposals, governors will be able to vary the work programmes for prisoners allowing them to find innovative ways of giving them the skills to cope when they leave and not commit crimes again. Having a justice system that works for the public good means aiming to make good neighbours and citizens out of offenders, not giving into the desire to retaliate. I look forward to seeing the progress the Lord Chancellor makes on prisons over the next few years. If he is as courageous and ambitious as he was in education and sounds so far on justice, then this could one of our greatest legacies.