by Dr. Endre Borbáth
Postdoctoral Researcher, Freie Universität & Wissenschaftszentrum Berlin für Sozialforschung
Szerk.: Abban bízva, hogy az érdeklődőket ez nem tartja vissza az olvasástól, csak angolul tesszük közzé a romániai elnökválasztási közvélemény-kutatásokról készült, eredetileg itt magyarul megjelent összefoglalónk második részét. A második rész is az openpolitics.ro oldallal közreműködésben készült.
In this article I follow up my pre-election review and compare the polling data collected there with the vote share each candidate received to assess how accurate polls were before the first round of the presidential elections. The comparison is made to the vote share gained within Romania because opinion polls do not cover those who live and vote abroad.
As the first figure shows, the polls accurately predicted the relative ranking of the major candidates. The largest error occurred in the case of Klaus Iohannis, who got about four percent less of the domestic vote than the trend revealed by the polls would have predicted. The popularity of Iohannis among diaspora voters brought his election results slightly closer to the polls than the figure below suggests, but with literally all polls suggesting that he would win more than he actually did, either a systematic polling error occurred in his case, or his support dropped quite a bit in the end for some mysterious reasons.
A closer look at Iohannis’ polling numbers reveal two rather curious patterns. First, compared to the predictions by established polling houses (see those marked with filled symbols in the figure), polls with the dubious origin – mostly just leaked about alleged private polls for parties (see not-filled symbols) -, the latter were more accurate and tracked more closely (with less variance) the trend revealed by all published polls. Indeed, their figures varied so little regarding Iohannis’ vote share as if they had no random sampling error. These patterns suggest that the leaked poll figures may follow the results of other studies in their predictions.
The data regarding the popularity of Dăncilă followed a less systematic pattern except for an upward trend over the campaign. IMAS tended to have lower estimates for Dăncilă and higher for Iohannis than other polling companies, while CURS did the opposite (and was thus ultimately more accurate re the two leading candidates vote share). SOCIOPOL tended to publish above-average estimates for both Iohannis and Dăncilă, while SOCIODATA’s figures hovered around the trend of all companies combined.
Barna’s poll numbers had a relatively high variance, little in the way of change over time, proved relatively accurate at the end, and showed no obvious sign of house effects (i.e. no company revealed a systematic trend to suggest higher or lower support for him than other pollsters did around the same time).
Diaconu’s numbers are also worth mentioning, given his rapidly falling numbers after announcing his candidacy. As in the case of Iohannis, leaked polls seem to have less variation regarding his standing than polls by established companies. Among established companies, the last IMAS poll stands out by massively over-estimating the popularity of Diaconu compared to other studies and the vote share he ended up receiving.
As a rough measure of accuracy, I calculate the standard deviation of each company’s predictions around the vote share received by the four leading candidates (Iohannis, Dancila, Barna, and Diaconu). This time I show the deviations from both the candidates’ domestic and total vote share (the latter also includes the 650,000 votes cast abroad).
As the figure reveal again, the leaked private polls, about whom we do not even know who did them or what sample they used, seem to be more accurate than the polls properly published by identifiable polling houses. Among the polling companies with a longer history, IMAS and SocioData were the least successful, and CURS and SOCIOPOL were the most successful. The studies by the former two deviate on average by 6-8 percent, while the predictions of the latter two deviate on average by 2-3 percent compared to the final vote share of candidates.
While overall these numbers seem fairly small, the fact that all polling companies were better in predicting the total vote share of the different candidates than their domestic vote share is intriguing since for purely methodological reasons the opposite could be expected. Unfortunately, the methodological choices different companies make about sampling and weighting are not transparent and information about them is not publicly available.
A natural comparison could be made with the polls conducted in the context of the 2014 presidential elections. Our review the polls of a similar period from the beginning of August until the first round of the 2014 presidential election reveals that back then the list of companies publishing poll numbers was very different, with only one of them being present in 2019 as well (SOCIOPOL). Even though some of the ones from 2014 are still working with political polls (e.g. IRES), they were conspicuously silent in the lead up to the 2019 presidential race. If we calculate the same measure as the one presented above for the poll numbers of the four most popular candidates in the 2014 race (Ponta, Iohannis, Tariceanu, and Udrea), the average deviation (prediction error) across companies for the four most popular candidates was only 2 percent, half of the 4 percent observed in 2019.
To sum up, the analysis reveals a mixed picture. On the one hand, the polls predicted the relative order of candidates and were thus informative for voters who may have wanted to vote strategically. On the other hand, the market for political polling is characterized by a changing cast of companies that does not help the establishment of track records and brand name credibility, leaked polling numbers from unidentified sources, and a severe dearth of information about methodological details. The comparison to 2014 also shows that these problems do not show fading away.
Vox Populi 