Lewis Baston is author of Reggie: The Life of Reginald Maudling and several books about British general elections. He is a consultant on politics, elections and constituencies.

It has not been a good couple of weeks for the reputation of Parliament and government, to put it mildly. Westminster scandals start in the most unexpected places, from the bankruptcy court in Wakefield in 1972 to Mohamed Al-Fayed’s offices at Harrods in 1993 and now to the exposure of the sordid Hollywood kingdom of Harvey Weinstein in 2017. Here we are again: a different issue, but one with a lot of similarities to three recent periods in which Westminster’s seedier side has come to public attention.

There have been four recent scandals about standards of conduct in Parliament that have raised systemic questions about the way the whole place works, rather than being isolated examples of individual misfortune or misconduct. The history of scandals and low parliamentary standards obviously goes back a lot further. We have the precedent of the 1920s sale of peerages, the flagrant bribery during the passage of the 1840s railway bills, the electoral corruption Dickens satirised as ‘Eatanswill’, the machinery of patronage and money that kept Walpole as Prime Minister for 21 years… but, for current purposes, let us start with the 1970s.

The first systemic scandal of our times – or at least, as a child of the Heath era, my times – was about lobbying, contracts, conflicts of interest and influence-peddling. It goes under the general title of ‘Poulson’, and a lot of it did revolve around the Pontefract architect, but it threw up a lot of other questions. There were MPs on all sides in the 1960s known for open lobbying: Jim Callaghan, as Chancellor, said across the floor of certain MPs that: ‘I have almost forgotten their constituencies, but I shall never forget their interests.’

The rules on disclosing interests at the time were largely left up to the individual conscience of MPs, and many fell short of the standards expected even then, let alone the tighter ones that we now expect. During the late 1960s, Gordon Bagier, a Labour MP, signed up as a parliamentary consultant for a PR agency and spoke up in support of the firm’s client, the fascist regime in Greece. There were inconclusive discussions about acknowledging and registering interests, but these lapsed with the 1970 election.

The problem returned with a vengeance with the bankruptcy hearings of John Poulson in 1972, at which evidence of payments and consultancies to a range of political figures was produced, and in the criminal trials that followed. Reggie Maudling was the only Cabinet casualty; on the face of it, his resignation in July 1972 was on the grounds that, as Home Secretary, he was in charge of the Metropolitan Police, who were investigating his erstwhile business partner. The full extent to which he was involved in Poulson’s corrupt activities overseas did not emerge until much later although, by 1974, it was clear that he had lobbied government for overseas development funds that would help a Poulson project in Malta, and extracted extensive personal benefits from his association with him.

The response to the Poulson scandal was fumbled; it was a lost opportunity to change parliament. The Register of Members’ Interests was introduced in 1974, but it was voluntary, only taken occasionally and negligently policed; many MPs made partial declarations, and an impression grew up that if there was some allusion to an interest on the Register, then it was automatically acceptable. The Royal Commission on Standards in Public Life of 1974-76 led nowhere. The Select Committee on the Poulson MPs reported in 1977, and censured John Cordle (who had boasted on paper that he had participated in a Commons debate mainly for Poulson’s benefit). He resigned, but the Commons failed to support even a slap on the wrist for the other two MPs, Maudling and Albert Roberts. This encouraged MPs during the buccaneering 1980s to take ever greater risks for money, encouraged by the seedier side of the growing public relations industry.

Failure in the mid-1970s therefore led to the next round in the mid-1990s, when there were overlapping sets of scandals – some around personal lives, encouraged by Major’s ‘Back to Basics’ speech of 1993 and a restive press, but others because of the re-emergence of the problems over conflict of interest and advocacy when the ‘Cash for Questions’ affair broke in 1994.

The response was less fumbled this time, with Lord Nolan establishing guiding principles of public life in 1994, followed by the 1995 code of conduct for parliamentarians, and the establishment of a more effective investigatory system under the Parliamentary Commissioner for Standards. Parliament arguably ended up over-reacting, with a ban on ‘paid advocacy’ which is drawn so widely that it does interfere with the ability of MPs to speak knowledgeably about foreign affairs in particular. But over-reaction is part of the pattern, and perhaps a necessary one; a minority bring an institution or a set of practices into disrepute, but the institution as a whole needs to pay a price. This episode did not directly bring about any Cabinet resignations, although the fall of David Mellor (September 1992) was a prelude to the personal scandals and the issues swirling around Jonathan Aitken (resigned 1995, jailed 1997) had some common themes to cash for questions in terms of declared interests, business abroad and dishonesty.

The gap until the next systemic parliamentary scandal was shorter – ‘expenses’ broke upon the scene in 2009, although it had been in the post for a while, as journalists sought disclosure using the Freedom of Information Act 2000: the Scottish Parliament had already been rocked by an expenses row in 2005 which caused the resignation of the Conservative leader, David McLetchie. As with the 1974 scandal, it was all the more lurid for being revealed against the background of economic recession and, like both predecessors, contributed to the impression of a ‘fag-end’ period of government and public life. Many in the press and public fell into the lazy temptation to conclude that ‘they were all at it’, and not distinguish properly between fraud, the abuse of a system that was open to abuse, minor transgressions and, in several cases, total innocence.

Although the scandal broke in 2009, which was the year in which Michael Martin, then the Speaker, stood down, and when MPs were most under pressure, the two Cabinet level casualties fell later. David Laws served as Chief Secretary to the Treasury for a short time in May 2010, before his expenses claims relating to his London flat proved difficult to explain to a public who were being invited to embrace austerity. Four years later, the then Culture Secretary, Maria Miller, eventually resigned after trying, with Downing Street support, to fight off criticism of her housing expenses that first came to light in 2012. As with the 1990s scandal about outside earnings, the expenses scandal resulted in a strong response from Parliament as an institution, which involved the establishment of IPSA and a regime of transparency.

The current one about sexual harassment has not yet run its course, although it has claimed one Cabinet casualty in Michael Fallon and, while allegations have been made against Damian Green he has denied them and retained his position, though there is an official investigation. As with other scandal-hit periods, other incidents and allegations seem to rain in from all sides, so the historical significance is a bit obscured by the sudden end to Priti Patel’s real-life Ugandan discussions. But the parliamentary sexual harassment scandal fits the ‘systemic’ criteria in that while it has affected the Cabinet it clearly goes wider than party. As with exploiting outside consultancies and imaginative expense claims, people on the inside have known for some time that it goes on, and have generally also known which individuals are most guilty, but little effort was made to do anything about the problem until it blewup in public.

There are some other common themes to these scandals. None have come out of a clear blue sky, and struck down a government that was in command of the political environment. In 1994 and 2009 governments were on the way out after a long period of office (the same can be said for the Profumo scandal in 1963, although that was less systemic and did not revolve around Parliament). In 1974 and 2017, governments lacked proper parliamentary majorities and faced daunting circumstances. Most scandal seasons implicate people in more than one party if they are truly systemic – the 1990s being an exception.

The response to systemic scandals has generally been to increase transparency, tighten the rules and set up new institutions, distanced from parliamentarians, to police them. It is hard to argue against the logic, but the process is not without its own problems. If revenge is wild justice, transparency can be wild scrutiny; while too often self-regulation has demonstrably failed, the alternative should involve the public in something better than indiscriminate condemnation. An environment in which there is a strict set of rules and an unforgiving attitude towards breaches of the letter of the rules can be a harsh one, but it is perhaps a just punishment for institutions (and the individuals that comprise institutions) when reliance on individual integrity has let down the interests of the public or of people who have to work alongside MPs.

In the best of all worlds, we would have a system under which standards are upheld voluntarily, with a shared ethos being understood by representatives and potential transgressions identified and corrected by colleagues before they become too serious. But in the world we have, we seem to need the rules, and occasional periods when there is a public need to recriminate and extend them. It’s not very nice out there in the pouring rain, which rains alike on the just and the unjust, but the place is cleaner and fresher after the storm.