(by Chris Hanretty) I’m more an end-user rather than a customer: I don’t buy opinion polls, but rely on a number of polling companies to publish their results which are (typically) commissioned by national newspapers or other media outlets.

This market has recently suffered a clamorous and highly visible failure. Most polls before the 2015 UK general election suggested that approximately equal proportions of people intended to vote for the Conservative and Labour parties respectively. In the end, the Conservatives finished six and a half percentage points ahead.

Given the size of this failure — and CCP’s obvious interest in competition and troubled markets — it’s reasonable to ask whether the polling industry’s failure is evidence of a broader failure in the market for public opinion research.

It’s helpful to characterise the market.

• public political opinion polling is not profitable, and is generally cross-subsidised by other market research operations.
• in one case (Lord Ashcroft polls) political opinion polling is carried out for free and provided as a public good.
• the set of potential customers is small, being restricted to national dailies and ITV (BBC tends not to commission straight-up vote intention polls)
• product quality is assessed periodically in national elections, and customers only receive a noisy signal of product quality: pollsters might be (in)accurate because they have good procedures which generally produce reliable results, or because the gods of sampling variation smiled on them in their final poll
• the barriers to entry are lower than they ever have been, thanks to the advent of online polling

This last point is particularly important. The following graph shows the number of active polling companies in each election since 1979. There were more polling companies in this election than at any previous election. It doesn’t seem that the polling industry deliver a poor performance because significant barriers to entry preserved “low-quality” pollsters. But if the 1992 experience is anything to go by – where the polling companies similarly under-estimated the Conservatives to quite a considerable degree – that number might shrink before 2020 as companies exit this tricky market.

The larger number of companies does make it harder to credit some of the accounts of herding that we’ve seen. The polling company Survation have already to pulling a poll because they thought it looked like an outlier. And Harry Enten has already that the variance of polls decreased in the last weeks of campaign — which might be evidence of voters being less erratic, or might be evidence that there was some tweaking in order to avoid ending up far from the herd. It would be quite impressive were herding dynamics to survive in a market with so many competitors — so maybe this just is a systematic failure.

What would make it easier would be if there were more frequent, more accurate signals of product quality. If the UK were to have elections every two years rather than every five, then we’d be able to get a far better grasp of which polling companies were good, and which merely lucky. In this case, however, the remedy might seem more extreme than the original malady…