Tuesday, 7 April 2009

Hung Parliament Revisited

A PB2 guest slot by Rod Crosby

Discussions of the prospect of a hung parliament have probably caused more disagreements here than any other topic. The preponderance of opinion seems to be that the Tories are very likely to win a majority. All things are possible, but before one commits to a position an objective analysis is called for.

What is the Electoral System saying?
The capacity of FPTP to deliver an overall majority is entirely dependent on two factors – its exaggeration when translating votes into seats, and the number of third party MPs.

The first has been in long-term decline since the demise of the “cube-law” in the 1950s, although it has recovered slightly since 1992 and has now stabilised at around a “square-law.” This change, due to the decline in the number of marginals, means that fewer seats now change hands between Labour and Conservative for a given shift in votes, or swing.

The second has seen the number of MPs not aligned with Labour or Conservative grow from just 8 in 1955 to no fewer than 92 in 2005. This change, due to the rise in the LibDems and Nationalists, and the de-coupling of Northern Ireland from the mainland party system, means that to gain an overall majority Labour or Conservative now need a far larger lead in seats over each other than they did in the past.



Combined, these two factors have dramatically altered the British FPTP system from one where hung parliaments were extremely unlikely to one where they are now increasingly likely.

The rĂ´le of Electoral Bias
The preceding analysis takes no account of electoral bias – the tendency of the system to favour one party over another, other things being equal. In the 1950s there was a modest pro-Tory bias, but by 1966 this had disappeared, and for the next 25 years the system treated the two big parties roughly equally. In 1992 that changed dramatically. Despite obtaining a thumping lead in votes, John Major was rewarded with a majority insufficient to last a full parliament. Tactical voting, and to a lesser degree regional effects, had swung the system significantly towards Labour. And rather than this being an aberration - on the contrary - the anti-Tory bias increased in 1997, 2001 and 2005. It is notable that, but for electoral bias, there would have been a hung parliament in 2005. Boundary changes and a likely small fall in the total number of third-party MPs may reduce this bias fractionally in 2010, but as the following graph shows, the Tories will probably need about 57% of the two-party vote to achieve a majority – a huge handicap only narrowly exceeded in 2005. However, continuing demographic shifts may also improve Labour’s position slightly, relative to the 2005 notional figures, meaning that in practice the Tories could be no better off than in 2005.



In a nutshell, the hung parliament zone has widened from a crack to a chasm, while the power of the system to propel a party across it has simultaneously weakened. Worryingly for the Tories, electoral bias has now shifted the chasm almost entirely into what would otherwise be their winning territory.

Cycles and Volatility
A complicating factor, in arriving at an estimate of the chance of a hung parliament is that the political pendulum does not move in a clockwork fashion. Parties have their great triumphs and abject disasters, and often remain in power for more than one term.


The black line represents an idealised electoral system, with no electoral bias and assumes a swing to the opposition at each election, adjusted for increasing volatility over time. The yellow line is more realistic, incorporating electoral bias and memory of the previous election result.

The first big leap in the chance of a hung parliament came at the February 1974 election, which did in fact produce one, although the objective chance, given a small pro-Labour bias and the closeness of the previous result, was somewhat smaller. It was the unexpectedly minuscule swing to Labour that in reality delivered that inconclusive result. For different reasons, two subsequent elections - October 1974 and 1992 - also came extremely close to producing hung parliaments.

Of course, politics often confounds even the best statistical estimates. In 1983, had there been a swing to Labour, a hung parliament was quite likely - instead there was a swing to the government, resulting in a Tory landslide. Again, in 1997, the system was primed to deliver a hung parliament, but the unprecedented, massive swing to Labour rendered the estimate moot. On the other hand, the graph shows that in 1987 and 2001 the Opposition were simply starting from too far behind to make a hung parliament at all likely, which turned out to be an accurate prediction. But we can see that in 2010 we are once again facing a “peak probability” of a hung parliament. The numbers say about a 67% chance, and we can be pretty confident there will be no swing to Labour to upset that estimate.

What are the Polls saying?
Since, in the final analysis, a hung parliament turns on a single seat we should not be surprised that individual polls give a Yes/No answer, rather than a Maybe. That may be misleading. Of the 335 polls since 2005 exactly 50% of them have indicated a hung parliament, while only 28% have indicated a Tory majority and 22% a Labour majority. That is an unprecedented number of polls indicating a hung parliament.

The following graph visualises the polls. About half the time, we see “islands” of majority government emerging from the Stygian waters of a hung parliament, only to be submerged again. The Tory archipelago seemingly peaked in 2008, and one could be forgiven for thinking they look poised to slip beneath the waves again sometime soon…




The median forecast of the polls in this parliament is Con 288, Lab 277, LD 47, which if reflected in the result could see Labour remain in office with the LibDems as Kingmakers.

Conclusion
In 1951, just a 1% swing saw a Labour majority government replaced by a Tory majority government. Electoral bias, increased third party MPs, and the changing nature of the system mean that to obtain a majority at the next election the Tories require at least a 6% swing, possibly more. Only one of the last sixteen elections has seen such a swing. The polls are narrowing - another fractional shift and the Tories fall back into the NOM zone. Whether you examine the electoral system or the polls, a hung parliament still appears to this poster to be the odds-on favourite.

Rod Crosby is a regular contributor to the main site - Politicalbetting.com

32 comments:

wibbler said...

First!

Niklas Smith said...

I think assuming that the Lib Dems would support a minority Labour government with fewer seats than the Conservatives is a mistake. Firstly, as I understand it the party with the most seats is conventionally given first shot at forming a government.

Secondly, the Lib Dems would be shackling themselves to an unpopular party whose leader (probably still Gordon Brown) would undoubtedly be in a precarious position after losing the election.

I think the most likely hung parliament would be one in which the Conservatives have the largest number of seats, or at the very least the greatest popular vote (sadly these two are no longer guaranteed to happen together with FPTP). In that case the best (and most likely) Lib Dem policy would be limited cooperation, but not coalition, with a minority government led by Mr Cameron. I would expect an agreement over some aspects of economic policy and maybe constitutional reform in exchange for a yes vote or abstention for the Queen's Speech.

John O said...

In this intimate corner, I must confess to be one of those in Mike's most recent 'poll' who plumped for the Conservatives being the largest party in a Hung parliament (much as I would want a landslide).

Which would win my bet with RodCrosby which I'm sure will happen.

I don't buy-in to his automatic swing-back theory - Labour could easily poll lower than currently: we just don't know at this stage.

And if, on the day, the Tories are over 40% and have a 10-12% lead, then I would bet on them having a workable overall majority.

Timothy (likes zebras) said...

I find it interesting that the previous time the chance of a hung parliament was similarly large was in 1997.

I also note that you say the electoral bias [which for you includes the effects of tactical voting] has increased in strength for Labour in the last few elections.

Many people are assuming that tactical voting will go in reverse, and this would presumably shift the chasm in the Tories favour, reducing the chance of a hung Parliament, in the expected scenario of a swing towards the Tories.

[The last two figures are the wrong way around]

Richard Nabavi said...

"Electoral bias, increased third party MPs, and the changing nature of the system mean that to obtain a majority at the next election the Tories require at least a 6% swing, possibly more."

I don't think anyone disagrees with that; the exact figure will of course depend on tactical unwind, but 6% is a reasonable guess.

It's this bit where Rod seems to me to go wrong:

"Only one of the last sixteen elections has seen such a swing. The polls are narrowing - another fractional shift and the Tories fall back into the NOM zone."

Just because swings of that magnitude don't happen very often doesn't mean such a swing won't happen this time. You might as well say 'In most years unemployment stays well below 3m' and conclude that it therefore won't reach 3m this year. In both cases, you need to look at the underlying driving forces to determine whether this time round will be one of the atypical cases.

And as for the polls - yes, of course a swing back to Labour would make a hung parliament more likely. But equally a small swing in the other direction would take us into Conservative landslide territory.

Looking at the underlying realities, the latter seems more likely.

Anonymous said...

I entered this market in the summer of 2006 with two firm convictions.
1.That LAB couldn't get an Overall.
2.There would be a Hung Parliament.
Among the cognoscenti and the 'cognoscenti' support for this view was not only unanimous but it was even more strident than my own.

The king of Betfair Politics opined that NOM was a true 1.30 shot.I put him in his place !
At this time the two main Parties were locked together Seat-wise on the Spreads and the auguries for a Hung Parliament could not have been brighter.It just seemed a question of waiting for polling day to collect.
How wrong we were !
Great thread and I will be a regular nuisance on it.You probably know who I am.

Anonymous said...

In the abstract, is Rod Crosby correct in saying that the chances of a hung Parliament at an election are greater now than they were 50 years ago? Yes, of course he is. We have a substantial third party and more minority parties.

Will it happen at the next election? That's a very different question and one which statistics can't answer alone.

The best analogy I can give is from cards. I like playing bridge. When you hold the king of a suit, you will without any further knowledge of the hand expect that it will take a trick more often than not. Perhaps three times out of four. But when you play a given hand, the long term odds are primarily of background interest only - you look at the hand as a whole, listen to the bidding and make your assessment in the light of all the information. The mistake that I perceive Rod Crosby to be making is that he is not looking at the hand as a whole and listening to the bidding.

antifrank

Mike Smithson said...

I'm not comfortable with Rod's selective use of polls to argue his case. To talk about the 335 surveys since 2005 is a nonsense and base predictions on that suggests that he has decided what he wants to prove and is looking for the numbers to support it.

Rod ought to concentrate on what's happened since October 2007 when Brown bottled it over a general election. That was when the latest political period began and a huge proportion of those surveys have pointed to s Tory majority.

I find some of Rod's number unfathomable. To talk with an apparent degree of certainty of a 67% chance of a hung parliament with current polling numbers is loopy.

Mike Smithson

Innocent Abroad said...

Many thanks, Rod.

I'm sure both Niklas and Richard are right in what they say, too.

However, given the scale of sleaze in this Parliament, I think there is also a possibility that Labour will simply implode. (Of course, sleaze isn't only a Labour problem.)

Anonymous said...

They say 'you gotta dance with the one you brung'.
I brung two !The first was that a Hung Parliament had never looked more likely....ever !
The second was a view that a Tory success would mean a partial eclipse for the Lib Dems.As you can see,these two dancing partners were not in step.
Exposure to pb.com and to Mike in particular has made me less strident about the fate of the Lib Dems, which should make me more bullish about the chances of an HP.

Rod Crosby has advanced two additional theories in favour of a Hung Parliament.....a demographic shift towards Labour and the magic 'swingback'.
I would counter sharply with the unwind and probable reversal of tactical voting(TV).

Anonymous said...

Roger , excellent thread, very interesting indeed.

MalcolmG

RodCrosby said...

The last two graphs are in the wrong order, Mike.

It's not nonsense, or selective, to look at all the data, Mike.

The approach here is twofold. The polls show great volatility, with the seat lead probably changing up to seven times during this parliament. The median seat figures are useful in that they are the levels that the parties have been above and below half the time.

The other approach is to measure the average swings to the opposition, controlled for increasing volatility (standard deviation) over time, and make an estimate of the chance the Tories fall in the NOM zone. That is where the 67% comes from.

These figures are just estimates, and it's quite possible for others to find different estimates using other methods. I'd be interested to hear of a different approach.

But to reject a mathematical approach out of hand is just unscientific.

I state that political events can of course confound these estimates, and give examples of this happening in the past. I cannot foretell the future, but I can demonstrate that, if you like, with "average luck" a hung parliament is rather likely, but, of course, by no means certain.

Jonathan said...

By all means use all the data. But really I can't see all that much point in proving that Tony Blair would be leading Labour into a hung parliament.

RodCrosby said...

Niklas Smith, "I understand it the party with the most seats is conventionally given first shot at forming a government."

Not so. Elections, per se, have no constitutional significance. It would be quite in order for Brown to continue in office and meet parliament. Only if he was defeated on the address, would he have to resign, like Baldwin in 1924. He could of course decide to resign immediately, like Baldwin in 1929, or cling on for a few days like Heath in March 1974.

Anonymous said...

To illustrate the dangers of becoming too dogmatic in this discussion,these are the LOWEST traded prices on the three options.

NOM 2.0
LAB Overall 1.62
CON Overall 1.50.
At some point or another,all three options have been hot favourite.
You could argue that now we are getting closer to the time of the next GE and therefore there is less scope for further twists.

Mike Smithson said...

I'm not use to this Blogger software. Does anybody know how to edit and save without creating a new post - which is what I keep on doing.

There does not seem to be a save edit option.

Mike Smithson

RodCrosby said...

"It's this bit where Rod seems to me to go wrong:"

What?

I've just stated facts. The Tories do need to exceed all but on of the last 16 swings, and the polls have narrowed.

Richard Nabavi said...

Rod - "I've just stated facts. The Tories do need to exceed all but on of the last 16 swings, and the polls have narrowed."

Yes, but you are wrong in the conclusions you draw from those facts. Your logic is that if a particular outcome has happened only once in the previous 16 times, it is unlikely to happen the next time. That is tosh, unless the events are randomly distributed. But they are not randomly distributed; they have causes.

As I pointed out: Unemployment has not exceeded 3m at any time in the last 20 years. By the same reasoning you apply to electoral swings, are you saying therefore that there is less than a 5% chance of it exceeding 3m in 2010? If not, why not?

As for the polls narrowing - yes, they have, somewhat, compared with the peak last August. That doesn't in itself mean that you can predict that they will narrow further. They might widen again (as they have been widening since December). In order to decide which direction is more likely, you need to assess the underlying causes of the changes, and make a judgement about their political effects. Historical statistics are of very little help in this judgement.

Anonymous said...

"But to reject a mathematical approach out of hand is just unscientific."

I'm sorry, this just won't do.

It's not the numbers (the mathematics, if you like) that make the election results but the results that make the numbers (maths).

Crosby is ALWAYS looking at the issue down the wrong end of the telescope.

It will be the aggregate of millions of individuals' behaviours that determine the result (and the numbers) in 2010 not the numbers from poll predictions, massaged by swing-back or whatever that determine the numbers. And then you'll need a new theory since those behaviours could add up to a variety of election results yet aggregate to a single, overarching set of election numbers.

The man is an obsessive idiot.

RodCrosby said...

The answer to that is that the unemployed don't consciously choose to be unemployed - it just happens, and there's no practical ceiling for it.
Voting, on the other hand is a conscious act, and one could make a strong case that there is a practical maximum in a two-party system - a party is unlikely to get more than about 60% of the two-party vote. The Tories need to be at the upper end of this figure, so it's a lot easier to measure the chance of this.

Got to dash out, otherwise would elaborate further.

Niklas Smith said...

Re. Rod: "Not so. Elections, per se, have no constitutional significance. It would be quite in order for Brown to continue in office and meet parliament. Only if he was defeated on the address, would he have to resign, like Baldwin in 1924. He could of course decide to resign immediately, like Baldwin in 1929, or cling on for a few days like Heath in March 1974."

Good point. However, ever since Disraeli resigned as soon as the election result was declared in 1868 (I think) it has become unconventional for a PM to hang on until he/she meets the new House of Commons. How much leeway does the Queen have in deciding who to ask to form a government first?

Anonymous said...

Rod. I am not a school teacher or an academic, but again I must correct your work. You are using the wrong word to describe the effect of changes in construction of individual constituencies. These changes are not demographic. Demography is births and deaths. To state that demography provides Labour with 2 seats a year implies that Conservative voters die at a quicker rate than Labour voters, which in turn would then imply that the Conservative party would eventually disappear. The effect that you described somewherelese of the phenomenon that you were discussing is internal migration - people moving out of small Labour held seats to larger Conservative held seats. To call these shifts demographic is to misunderstand what the word means. Please can you correctly describe them in future then everyone will know what you mean.

Anonymous said...

10.37AM 11.10AM and the one with the three betting figures was me.
I now on reflection feel it dishonourable to hide behind the cloak of anonymity.

URW

RodCrosby said...

Er?

http://en.wikipedia.org/wiki/Demography

RodCrosby said...

"it has become unconventional for a PM to hang on until he/she meets the new House of Commons."

It has become rare, simply because hung parliaments hitherto have been rare. But it is entirely "constitutional" to do so.

The Queen's powers are limited by convention also. The reserve powers are just that, only to be used in extremis, if a politician in turn tries to do something "unconstitutional." Waiting to meet parliament is entirely proper, although it may not be politically wise - but that is not a constitutional question.

The classic example is Mackenzie King in Canada in 1925. He emerged from the election having lost his majority and his own parliamentary seat, but carried on as PM as leader outside parliament of the second largest party in parliament. The same thing or a variation could happen here.

The Queen would only get involved if a PM who had requested a dissolution subsequently failed his first test in the House, and asked for another dissolution. She would refuse, effectively dismissing him. The PM knows that, so wouldn't ask...

RodCrosby said...

"It's not the numbers (the mathematics, if you like) that make the election results but the results that make the numbers (maths)."

By the same logic, all previous election results should be discarded. Everything starts de novo. So the Tories can win Bootle just as likely as they win Buckingham.

If that is the case, dare I ask: Why are you here at all?

The truth is yes, the Tories can win Bootle, but with a vanishingly small probability that can still be estimated.

The chance of a hung parliament can be estimated also.

Mike Smithson said...

Rod - You say ". However, continuing demographic shifts may also improve Labour’s position slightly, relative to the 2005 notional figures, meaning that in practice the Tories could be no better off than in 2005."

Eh? So you are suggesting a repeat of the 2005 election result. Come on! I hear much of the case that you are making but I simply cannot see any foundation for much of it.

Your good arguments are being drowned out by the rubbish.



Mike Smithson

RodCrosby said...

Space limitations prevent me from spelling out each and every step of my analysis, Mike.

What I meant was, demographic changes may attenuate the net notional gains the Tories have made as a result of the boundary review, leaving their starting position closer to the actual 2005 result, rather than the notional 2005 result.

It is worth noting that, even without this factor, no less a person than Robert Waller has described the Tory notional gains as "illusory"...

Andy Cooke said...

I agree with the width of the chasm - the "Non-Big-Two" effects are indisputable.

However, Rod points out that the biggest single factor in tilting the system against the Tories (the movement of the chasm into the Tory lead area) has been tactical voting - "Anyone but the Tory". I'd add that Blair's ability to maximise the vote share in the swing voting middle classes (he provided a boost in the marginals over the safe seats) added to this distortion.

In order for the tactical voting distortion to continue (and thus that (largest) component of the pro-Labour bias to continue), the "Anyone-but-the-Tory" drive must continue at the same level. Increase in that drive further tilts the landscape pro-Labour; any decrease acts towards levelling it back off.

The Blair appeal to the marginals would have to be replicated by Brown or ground would be lost in that (lesser) effect as well.

I'd imagine that although these two effects aren't mathematically modelled, an educated opinion as to their strengths could be gleaned from scrutinising the "Forced choice" and class-breakdown of the polls. The informed punter can then choose how to adjust the chasm to suit their personal opinion. I'd note that a full unwind wouldn't actually position the chasm centrally - the tendency of Labour seats to be smaller coupled with the traditionally lower turnout in labour safe seats would (in my opinion) put the "total unwind" position about two-thirds of the way back from the current position as of 2005 and the dead centre.

Andy Cooke

Anonymous said...

Ok Rod I'm busted. Beside being trained in statistics I am also trained in demographics. So that makes me an actuary although lapsed since I have not prcaticed for many years.

Your use of Wikipedia just shows two facts. Wikipedia is not the font of all knowledge, just one person's view until someone else can be bothered to correct it. Secondly, context is everything.

When I studied my text books on demography, mortality and births were the major issue. Sure migration was considered as a secondary effect when projecting a country's population, but your comments have referred to "demographics". The effect you wnated to describe would have been more accuarately described as "internal migration" so as to exclude births and deaths.

Richard Allen said...

Frankly I just don't see why anyone would regard polls taken in 2005 to have any useful insight into a 2009/2010 election. The political climate has changed so much since then that you might as well look at polls from 1905.

That is not to say that there isn't some merit to Rod's arguement but taking polls over the full course of the parliament makes no sense to me.

RodCrosby said...

Richard Allen. It's not as if I'm giving them undue weight. The polls have fluctuated dramatically, and there's no reason why they can't change again.

For half the time a hung parliament has been forecast. That is notable in itself, and no previous parliament can provide an example.

I'm just providing information, and it's up to the individual to make an educated guess what will happen.