Some Microfoundations of Collective Wisdom

Tuesday 18 May 2010, by Lu Hong, Scott E. Page

Version imprimable fontsizeup fontsizedown
Télécharger cet article au format PDF (151.2 kb)

Abstract

Collective wisdom refers to the ability of a population or group of individuals to make an accurate prediction of a future outcome or an accurate characterization of a current outcome. Without feedbacks, the collective will always be more accurate than it’s average member, and in some circumstances, it can be more accurate than any of its members. Yet, collective wisdom need not emerge in all situations. Crowds can be unwise as well as prescient. In this paper, we unpack what underpins and what undermines collective wisdom using a model of agents with predictive models. Our model extends traditional statistical approaches to characterizing collective wisdom. Within our model, we demonstrate how collective accuracy requires either individual sophistication/expertise or collective diversity. A lack of both characteristics necessarily leads to breakdowns in collective wisdom.

In describing the benefits of democracy, Aristotle observed that when individuals see distinct parts of the whole, the collective appraisal can surpass that of individuals. Centuries later, von Hayek in describing the role of information in decentralized markets made a related argument that suggested the market can accurately determine prices even if the average person in the market cannot (von Hayek 1945). To be sure, institutional structures such as democracies and markets rests substantially on the emergence of collective wisdom. Without a general tendency for groups of people to make reasonable appraisals and decisions, democracy would be doomed. The success of democracies, and for that matter markets, provides broad stroke support that collective wisdom often does exist. Abundant anecdotal and small to large scale empirical examples also suggest at least the potential for a ”wisdom of crowds” (Suroweicki 2004).

Collective wisdom, as we shall define it here, exists when the crowd outperforms the people in it at a predictive task. This is a restrictive notion. Wisdom often has a broader conception than mere accuracy. A society deliberating on laws or common purpose must exercise wisdom in judgement. The task is much richer and nuanced then estimating the value of a stock or the weight of a steer. And yet, if we see wisdom in these contexts and anticipating multiple implications and interactions, then we might also wisdom as the ability to recognize the multiplicity of effects and to accurately predict the magnitude of each. If so, we might see our conception of collective wisdom as less circumscribed.

The logical statistical foundations for collective wisdom are well known. First, a straightforward mathematical calculation demonstrates that the average prediction of a crowd always outperforms the crowd’s average member (Page 2007). Second, this same calculation implies that with some regularity, crowds can outperform any member or all but a few of their members. We describe how that can be the case in detail.

Mathematics and lofty prose not withstanding, the claim that the whole of a society or group somehow exceeds the sum of its parts occurs to many to be over idealized. Any mathematician or philosopher who took a moment to venture out of his or her office would find no end of committee decisions, jury verdicts, democratic choices, and market valuations that have proven far wide of the mark. Collective wisdom, therefore, should be seen as a potential outcome, as something that can occur when the right conditions hold, but it is in no way guaranteed.

The gap between theory and reality can be explained by the starkness of existing theory. The core assumptions that drive the mathematical necessity of collective wisdom may be too convenient. In particular, the idea that people receive independent signals that correlate with the truth has come to be accepted without thought. And, as we shall argue, it is this assumption that creates the near inevitability of collective wisdom.

In this paper, we describe a richer theoretical structure that can explain the ex- istence of collective wisdom as well as the lack thereof. In this model, individuals possess predictive models. Hong and Page (2007) refer to these as interpreted signals to capture the fact that these predictions can be thought of as statistical signals but that their values depend on how people interpret the world. the prediction of a crowd of people can be thought of as some type of average of the models contained within those peoples heads. Thus, collective wisdom depends on characteristics of the models people carry around in their heads. We show that for collective wisdom to emerge those models must be sophisticated, or they must be diverse. Ideally, crowds will possess both.

These two features refer to different units of analysis. Diversity refers to the collection seen as a whole. The people within it, or their models, must differ. Sophistication /expertise refers to the capabilities of individuals within the collection. The individuals must be smart. There need not be a tradeoff between these two aspects of a crowd. Crowd members can become both more sophisticated and more diverse. They can also become less sophisticated and less diverse. In the former case, the crowd becomes more accurate, and in the latter case, they become less so. A tradeoff does exist in the necessity of these characteristics for an accurate crowd. Homogeneous crowds can only be accurate if they contain extremely sophisticated individuals, and groups of naive individuals can only be collectively accurate if they possess great diversity. [1]

The intuition for why collective wisdom requires sophisticated individuals when those individuals are homogeneous should be straightforward. We cannot expect anintelligent whole to emerge from incompetent parts. The intuition for why diversity matters, and matters as much as it does, proves more subtle, so much so that several accounts misinterpret the mechanism through which diversity operates and that others resort to hand waving. The logic for why diversity matters requires two steps. First, diverse models tend to produce negatively correlated predictions. [2] Second, negatively correlated predictions produce better aggregate outcomes. If two predictions are negatively correlated when one tends to be high, the other tends to be low, making the average more accurate.

The model we describe differs from the standard approach in political science and economics, or what we call the statistical model of aggregation. As mentioned above, in the statistical model, individuals receive signals that correlate with the value or outcome of interest. Each individual’s signal may not be that accurate but in aggregate, owing to a law of large numbers logic, those errors tend to cancel. In the canonical statistical model, errors are assumed to be independent. More elaborate versions of the model include both negative and positive correlation, a modification we take up at some length as negative correlation proves to be crucial for collective wisdom.

In what follows, we first describe the statistical model of collective wisdom. This approach dominates the social science literature on voting and markets as well as the early computational literature on ensemble learning. That said, the computational scientists do a much more complete job of characterizing the contributions of diversity. Social science models tend to sweep diversity under the rug – calling it noise. In fact, we might even go so far as to say that social scientists consider diversity to be more of an inconvenience than a benefit.

We then formally define interpreted signals (Hong and Page 2007). These form the basis for what we will call the cognitive model of collective wisdom. This approach dominates the current computational science models. This cognitive model does not in any way contradict the statistical model. In fact, we rely on the statistical model as a lens through which to interpret the cognitive model. In characterizing both types of models, we consider a general environment that includes both binary choice environments, i.e. simple yes or no choices, and cardinal estimation, such as when a collection of people must predict the value of a stock or the rate of inflation. When necessary for clarity, we refer to the former as classification problems and to the latter as estimation problems. The analysis differs only slightly across the two domains, and the core intuitions prove to be the same. We conclude our analysis with a lengthy discussion of what the theoretical results imply for the the existence or lack thereof of collective wisdom in markets and democracies and we discuss what we call the paradox of weighting. That discussion is by no means exhaustive, but is meant to highlight the value of constructing deeper micro foundations.

Before beginning, we must address two issues. First, a growing literature in political science and in economics considers the implications of and incentives for strategic voting. For the most part, we steer clear of strategic considerations. When they do come into play, we point out what their effect might be. We want to make clear from the outset that regardless of what motivates the votes cast, the possibility of collective wisdom ultimately hinges on a combination of collective diversity and individual sophistication /expertise.

Second, we would be remiss if we did not note the irony of our model’s main result: that collective wisdom requires diverse or sophisticated models. Yet, in this paper, we have constructed just two models - a statistical model and a model based on individuals who themselves have models. If our theory is correct, these two cannot be enough. Far better that we have what Page (2007) calls “a crowd of models.” Complementing these models with historical, empirical, sociological, psychological, experimental, and computational models should provide a deeper, more accurate picture of what conditions must hold for collective wisdom to emerge. Clearly, cultural, social, and psychological distortions can also bias aggregation. We leave to other papers in this volume the task of fleshing out those other perspectives on collective intelligence. We note in passing that historical accounts, such as that of Ober in this volume, also identify diversity and sophistication as crucial to the production of collective wisdom.

Télécharger l’article complet/Download the paper : ici/here

by Lu Hong, Scott E. Page

Version imprimable fontsizeup fontsizedown
Pour citer cet article :
Télécharger cet article au format PDF (151.2 kb)

Footnotes

[1] Our approach borrows ideas from ensemble learning theory. In ensemble learning, collections of models are trained to make a prediction or a classification. The predictions of the individual models are then aggregated to produce a collective prediction.

[2] In the case of a yes or no choice, Hong and Page (2007) show that when people use maximally diverse models (we formalize this in the paper), their predictions are necessarily negatively correlated.

 

© Raison-Publique.fr 2009 | Toute reproduction des articles est interdite sans autorisation explicite de la rédaction.

Motorisé par SPIP | Webdesign : Abel Poucet | Crédits