Oh the joys of righteous indignation. Maria Miller has behaved no worse than hundreds of other MPs. But the evidence about her expenses claims lies ready to hand, and she has made an inadequate apology.

So let us hunt down Miller. Let us with whoops of joy fall upon this easy target. In the name of a higher morality, let us bully her out of public life.

The higher morality is meant to prevent people fiddling their expenses. In the old Fleet Street, the fiddling of expenses was not merely endemic, but was considered a matter for jovial boasting. Newcomers were taught how to do it, and would be reprimanded for letting the side down if they omitted to do it.

Ah, the moralists reply with ineffable smugness, but Miller was stealing public money. I would have thought theft is theft, whether the money belongs to Lord Copper or the Treasury. But I do not think “theft” is the right word here. Miller was claiming money to which she believed herself to be entitled. The overwhelming likelihood is that she was encouraged in this belief both by other MPs and by the officials who were running the system.

The moralists denounce Miller’s sense of entitlement, but never pause to wonder how entitled they are to try to destroy her. One day they compete to see who can inflict the greatest pain on this hapless woman and her family, the next day they assail Cameron for employing too many Etonians and too few women.

In December the Observer commissioned me to write a profile of Miller. It is not a very charitable piece: I quoted some unkind remarks about her, and myself described her as “unreportably dull”. It included the following information about her life before her election as MP for Basingstoke in 2005:

“Miller was born in Wolverhampton, but grew up at Bridgend, in South Wales, where she attended Brynteg comprehensive school, after which she read economics at the London School of Economics. She is not some glamorous creature who got parachuted in to a safe seat without first working her passage. She has belonged to the Tory party since 1983 and in the 2001 general election fought and lost Wolverhampton North East. Since 1990, she has been married to Iain Miller and together they have three children.”

She had no hope of winning Wolverhampton North East: Ken Purchase, the Labour incumbent, had over the course of several elections built up an impregnable majority. It must have been tough to leave three young children in London and go off to fight a losing battle in the Midlands.

But fortunately, Miller’s parents, John and June Lewis, were able to move from South Wales to the Millers’ home in Wimbledon, in south London, and look after the children. This sounds like an excellent arrangement. It is certainly one to which many women who wish to pursue a career have had to resort, as the best of the possible options. Yet the presence of her parents in the family home is taken as proof of Miller’s depravity.

If one takes a narrow view of this case, it is easy enough to find Miller guilty. But to take a narrow view is unfair. Here is a dutiful woman who for many years has worked hard to make her way along a path which very few women with young children have yet managed to follow. She may deserve reprimand, but she does not deserve persecution. We imagine ourselves to be living in a merciful age, but who has shown mercy to Miller? Yet it is hard to think of an offender who is less likely to repeat the offence. We prate about rehabilitation without actually practising it.

The resources of the moralists are not yet exhausted. If one demonstrates the unfairness of picking on Miller, they take refuge in denunciations of the entire political class. They place themselves at the head of a mob which would happily sweep away the whole corrupt edifice of Parliament. Let us have our version of the French Revolution. It will make us so much more pure.

If one opposes mob rule, and is dismayed by bullying, one cannot go along with these cheapskate moralists and their cost-free denunciations. The treatment of Miller has got completely out of hand. I am delighted that David Cameron has so far stood by her. The Prime Minister cannot govern by yielding to whichever moral spasm is currently convulsing the press. He has to take a wider and more balanced view. What woman with children would contemplate trying to become a Conservative MP if he throws Miller to the wolves?