For an entire season the researchers asked highly-knowledgeable fans of the (American) National Football League to make regular predictions about the outcomes of games. Some fans were asked to place bets against a points spread; for example, with a four-point spread against the Washington Redskins someone placing a bet on that team would only win if the Redskins won by a margin of more than four points. Someone betting on the Redskins' opponents would win if that team won or if the Redskins won by four points or less. Crucially for the purpose of this study, the points spreads used were increased by the researchers in an attempt to reduce or eliminate any advantage to betting on the favourite. This manipulation worked; against the points spread favourites lost more games than they won.
Other fans were asked to place a different, but logically-equivalent bet; they were asked to predict the winning team and to estimate the margin of victory. They received payments that depended on their accuracy. A third group of fans betted using a combination of the other two conditions. They first made a prediction against the points spreads, and then predicted who would win and by what margin.
In their main analyses, Simmons et al took the crowd judgment to be those cases where the proportion of money bet on the favourite was more, less, or equal to 50%. Contrary to the wisdom-of-crowds hypothesis, the crowd put its money on the favourite in 89% of games (thus lost money most of the time) and the crowd was worse than 93% of its members. For a subset of fans who had been warned that the spread had been adjusted, the crowd bet on the favourite only slightly less often (82.7% of the time) and still lost more often than they won. When people bet on favourites they placed more money than when they bet on underdogs, which suggests that they believed that the favourite would beat the points spread.
Was it the case that wisdom increased with experience? Did fans bet more appropriately with the passage of time (and experience of feedback)? No - as the season progressed they increasingly placed their money on favourites, thereby increasing their losses.
By contrast, those fans who were asked to predict the winner and estimate the points difference did much better. The crowd bet on the underdog 82.7% of the time, won against the spread 55.4% of the time, and outperformed 95.6% of its members. There was no change in performance over time.
The third group of fans, those who bet against the spread and predicted the victor/margin of victory, did just a little worse than those who only did the latter, but showed no overall bias towards betting on the favourite (unlike those who only bet against the spread).
Simmons et al explain these results in terms of their theory of intuitive confidence (Simmons and Nelson, 2006; see Chapter 15 in Hardman, 2009). They suggest that fans' intuitions about who will win leads them to underweight the points spread that they are presented with. However, the biasing influence of their intuition is attenuated when they themselves are required to think about what the points spread should be.
The overall message of the paper is that the extent to which crowds may or may not be wise depends on the way in which individuals are required to make their judgments.
This research was reported at the 2009 meeting of the Society for Judgment and Decision Making, Boston, USA. The working paper is:
Simmons, J.P., Nelson, L.D., Galak, J., and Frederick, S. (April 2009). Are crowds wise when predicting against points spreads? It depends on how you ask.
References
Simmons, J.P., and Nelson, L.D. (2006). Intuitive Confidence: Choosing Between Intuitive and Nonintuitive Alternatives. Journal of Experimental Psychology: General, 135, 409-428.
Posts
No comments:
Post a Comment