Lewis Baston is author of Reggie: The Life of Reginald Maudling and several books about British general elections. He is a consultant on politics, elections and constituencies.

This month has seen two Parliamentary by-elections. I write this column during the gap between between the by-election in Richmond Park (won by the Liberal Democrats) and the one in Sleaford & North Hykeham (retained by the Conservatives).

Although both seats were Tory in the 2015 general election, they are worlds away from each other, and on opposite sides of the political chasm that has opened up between the pro-Leave territory of most of England and pro-Remain London. Both by-elections, however, came as a result of principled decisions by their MPs arising from their opposition to important aspects of government policy – something that is comparatively rare in parliamentary history.

The Richmond Park case was particularly unusual because it involved an MP quitting, and then re-contesting the seat to test public opinion. Zac Goldsmith’s resignation came as a result of a specific promise made to his constituents about the issue of Heathrow expansion.

Since 1918, there have only been 24 by-elections caused by an MP resigning his or her seat to test opinion or validate a change of party allegiance – and 15 of these were in one batch in 1986 when Ulster Unionist MPs resigned their seats to demonstrate opposition to the 1985 Anglo-Irish Agreement signed by Margaret Thatcher and Garret FitzGerald.

Other than in this Unionist mini-general election, there have thus been nine by-elections of this kind (plus the Gravesend contest which was overtaken by the calling of a general election). Six have been to ratify a change of party allegiance, four of them successful. Four have been to test public opinion on a specific issue: the Duchess of Atholl in opposition to appeasement and the Munich agreement in 1938; Sir Richard Acland in 1955 over nuclear weapons, and the recent cases of David Davis over civil liberties in 2008 and Zac Goldsmith over Heathrow in 2016.

Davis is the only one to have succeeded – and, not coincidentally, the only one to have retained his official party nomination, and to have avoided competition from the other major parties. By-elections on specific issues have otherwise involved the MP concerned losing their official status and standing as an Independent.

Fraser Nelson has proposed “Zac’s Law” to prevent what he calls ‘hissy fit’ by-elections: an MP resigning his or her seat would not be permitted to stand in the ensuing contest. It is an unusual argument – one more frequently sees the case made that MPs who change their political colours during a parliament should face a by-election to obtain their electors’ permission to stay on.

Douglas Carswell based his decision to call a Clacton by-election in 2014 on this principle, although arguably going from member of the Conservative backbench awkward squad to rebel in a parliamentary party of one did not mark a dramatic change. While respecting Carswell’s view on the duties of MPs, I personally do not share it, and do not regard it as a strong precedent. Many MPs have changed allegiance in the past hundred years, but only ten have called by-elections as a result. Parties can change: many of the Labour to SDP defectors in 1981-82 regarded their new party as more faithful to the 1979 manifesto on which they had been elected than the old one.

The 2015 parliament has also seen wrenching change in the orientation of both main parties, and it would not be unreasonable for Labour social democrats or Conservative pro-Europeans to feel that they can jump ship without betraying what they said to the voters in 2015. While I am sympathetic to Nelson’s argument, I am not convinced that it is yet a sufficiently large problem to merit a new rule. And as he notes, the experience of the Richmond Park contest may be deterrent enough.

Resignation on an issue of principle, or indeed on grounds of general disaffection from party or government, are rare as well. These are harder to categorise because there may be cases where this consideration overlaps with more mundane reasons for by-elections such as family and career considerations.

Before Phillips this year, the last clear cut case of chucking it in for fundamental disagreement with the party was Bob Mellish in Bermondsey in 1983, and then before him Ray Gunter in 1972. There were a couple of borderline cases – that of Bryan Gould in 1994, who took an academic appointment in New Zealand but who had been increasingly unhappy with Labour’s approach to economics and the European Union; and Robert Kilroy-Silk in 1986, who had serious problems with his constituency party and left the Commons to become a television presenter. Other resignations to take other posts, but pushed along by policy disagreements, include Richard Marsh who was appointed Chairman of British Rail in 1971, and Alf Robens who went to the National Coal Board in 1960.

This month’s odd couple of by-elections comes against a background of a declining total number of by-elections, and some shifts in what has caused them to take place. Some causes that were commonplace in previous decades have either vanished completely or become rare. Before 1927, it was the rule for at least some ministerial offices that being appointed to the Government resulted in a by-election in the MP’s constituency – something which would have an interesting effect on the frequency and nature of reshuffles if it were to be revived.

Another defunct cause of by-elections is succession to a hereditary peerage. Until 1999 an occupational hazard, so to speak, for MPs from political dynasties was the unwelcome peerage. The most famous of these cases was that of Tony Benn, who succeeded to the Stansgate title in 1961, and was thus disqualified from office despite winning a by-election in Bristol South East after his unwilling elevation. Benn successfully campaigned for the right to disclaim peerages, and returned to the Commons in 1963, with Malcolm St Clair, the temporary Conservative MP for Bristol South East, doing the gentlemanly thing and standing down from Parliament to enable Benn’s return. Until Richmond Park, this was the last case of the Conservatives failing to defend one of their seats falling vacant.

Appointments to life or hereditary peerages have also become much less frequent, partly because by-election voting behaviour has become so volatile since the 1960s, and there is a presumption against causing unnecessary contests. There have only been four since 1979. The last was Hamilton in 1999, when George Robertson took a peerage and became Secretary-General of NATO. Before that there was Ribble Valley in 1991, when John Major put David Waddington in the Lords, and then Penrith and the Border in 1983 when Willie Whitelaw went to the red benches. Governments have tended to do badly in these elections: Hamilton and Penrith were held despite big swings against the government, while Ribble Valley and Workington (1976) were lost on huge swings. Becoming a television presenter is apparently much less of a risk: the vacancies generated by Robert Kilroy-Silk, Matthew Parris and Brian Walden were all successfully defended despite difficult political circumstances.

A constant occasional cause of by-elections has been gaining some other public office. Formerly, these were often nationalised industry, quango or judicial offices, and there is still the occasional diplomatic appointment (such as that of Sir Alastair Goodlad in 1999). The creation of the Scottish Parliament and Welsh Assembly have led to several of these, as have other new offices, such as Police and Crime Commissioners and directly elected mayors (Henley 2008, Leicester South 2011 and Tooting 2016).

Resignation following censure, public scandal or criminal conviction has become more common in the last ten years, with the expenses scandal causing a spate of these. It was previously a rare occurrence, although behind a select number of quiet, sudden resignations there has always been the breath of scandal. But vacancies in Beckenham (1997), Bournemouth East (1977) and Berwick-upon-Tweed (1973) were all under the shadow of scandal or censure. Before Barnsley Central in 2011, the previous by-election related to criminal conviction was Walsall North in 1976, caused by fraudulent long-distance swimmer John Stonehouse.

Mention of Stonehouse reminds me that the play This House, which I mentioned in passing in another history column, has made a welcome return to the stage, at the Garrick Theatre. Its core is the struggle of the Labour whips’ office to keep the show on the road from 1974 to 1979, and their relationships with their wayward flock and with their Conservative counterparts. It is an intriguing subject, particularly for those such as myself who are interested in the history of the 1970s, and those who lived through those turbulent parliaments: among my fellow theatre-goers was Lord Lamont, elected in a 1972 by-election for Kingston (including part of what is now Richmond Park). As This House demonstrates, the mortality rate among MPs was high between 1974 and 1979, with 13 Labour MPs dying in office, but surprisingly not quite as high as during 1966-70, when 20 died.

There was a big fall in the frequency of by-elections after the 1979 general election, for several reasons. One is that far fewer MPs die in office. Following the experiences of 1966-70 and 1974-79, when governments lost strings of by-elections, the parties are more likely to encourage elderly or unwell MPs to retire at general elections rather than soldier on, as Alfred ‘Doc’ Broughton, one of the heroes of This House, did in 1974-79.

The general fitness of MPs has improved, although that is difficult to quantify exactly. Labour’s working class MPs, particularly miners, tended to enter Parliament relatively late in life. They were sometimes suffering from the effects of long, arduous working lives; coal mining in particular took a physical toll on MPs, and the class inequality in life expectancy was reflected in the Commons. With the decline of dangerous occupations, and the changing social composition of the House so that the two parties are both relatively middle class, this has diminished. The lifestyle of MPs is also healthier, in the same way that the lifestyle of many people has generally improved. MPs are less likely to smoke and are less exposed to second-hand smoke in their workplace. They also, for the most part, drink less than they did in the 1970s. The whips’ offices of today are very different from their smoke-filled, whisky-fuelled, masculine predecessors from the 1970s, and there has been a certain amount of changing places – the Conservatives have had a former miner (Patrick McLoughlin) and Labour has had a ‘sort of’ advertising man (Nick Brown) as Chiefs since 2010.

But the words they will have had for awkward MPs who cause by-elections for some high-flown principle will have been much the same as the curses Phil Daniels spits out in his glorious This House portrayal of Labour’s 1974-76 Chief Whip, Bob Mellish.