Cultural Consensus Theory: Comparing different concepts of cultural truth

Abstract

Cultural Consensus Theory (CCT) is a model-based approach to aggregating the responses of informants (respondents) to questions (items) about some domain of their shared cultural knowledge. The purpose of CCT is to allow a researcher to discover consensus knowledge in cultural groups. This paper compares and contrasts two CCT models for items requiring a dichotomous, true/false answer. The first model is the General Condorcet Model (GCM). A special case of the GCM is already in wide use, especially in cultural anthropology, and this paper generalizes that version and provides new mathematical and statistical results for it. The character of the GCM is that of a general signal detection model, where the item-trial type (signal or noise) as well as the respondents’ hit and false alarm rates, are latent rather than observable. The second model, the Latent Truth Model (LTM), is a new model that allows cultural truth to assume continuous values in the unit interval rather than the two-valued truth assumption of the GCM. Both models are compared analytically, and hierarchical Bayesian inference for each is developed. A posterior predictive model check is established for both models that bears directly on the assumption that there is a single consensus truth. In addition, the similarities and differences between the models are illustrated both with mathematical and statistical results, as well as by analyzing real and simulated data sets, and a second posterior predictive check that tends to differentiate the models is also provided.

Introduction

Cultural Consensus Theory (CCT) is an approach to information pooling (aggregation, data fusion) first presented in the mid-1980s (Batchelder and Romney, 1986, Batchelder and Romney, 1988, Batchelder and Romney, 1989, Romney et al., 1987, Romney et al., 1986). Since its inception, CCT has become a popular methodology for measurement and inference in the social and behavioral sciences, especially in cultural anthropology (e.g., Romney & Batchelder, 1999). CCT is designed for situations in which a researcher has access to a group of informants (respondents, experts) who share knowledge or beliefs and agree to provide answers to a set of questions (items) about some domain of their shared knowledge. While the researcher is assumed able to pose relevant questions to the informants, the nature of their shared knowledge, and more specifically the consensus, or ‘culturally correct’ answers to the questions, are unknown a priori to the researcher. Thus the primary goal of CCT is to infer consensus patterns of cultural truth that represent the informants’ shared cultural knowledge. To achieve this goal, CCT consists of cognitive response models, each of which is designed to accommodate a particular testing format (e.g. true/false, multiple choice, ranking, probability estimates). Each CCT model specifies parameters for the consensus answer to each item, as well as for the response characteristics of each informant.

In the case of dichotomous true/false items, the standard CCT model is the General Condorcet Model (GCM), first published in the mid-1980s (Batchelder and Romney, 1986, Batchelder and Romney, 1988, Romney et al., 1986), and it is described later in this paper. The GCM has become the most popular model used in applications of CCT to ethnographic studies in cultural anthropology, e.g. Weller (2007), and there are also the many articles that cite Romney et al. (1986). The popularity of the GCM is due in part to the availability of a software package for estimating a restricted case of the GCM developed by Steve Borgatti, which at the time of writing, is available in ANTHROPAC (Borgatti, 1996) as well as in UCINET (Borgatti, Everett, & Freeman, 2002). The original estimation method is based on an ad hoc blend of method-of-moments and Bayesian techniques developed in Batchelder and Romney (1988), and it applies only to a special case of the GCM. Later Bayesian estimation for a more general version of the GCM based on Markov Chain Monte Carlo (MCMC) methods is presented in Karabatsos and Batchelder (2003). One of the main goals of this paper is to provide new mathematical results as well as Bayesian hierarchical inference for the GCM, including a new posterior predictive check for the central assumption that the informants share the same basic cultural truths.

In general, questions appropriate for CCT are designed to tap knowledge or beliefs shared by the informants rather than their personal beliefs or opinions. For example, (1) “Is Indianapolis the capital of Indiana?” and (2) “Do Hoosiers like basketball?” (Hoosiers are folks living in Indiana) are appropriate CCT questions; but (3) “Do you like basketball?” is not appropriate because it invites an individual to express a personal preference. The GCM assumes that each question pertaining to the shared knowledge of the informants actually has a dichotomously-based ‘correct answer’, i.e. it is either true or false for the culture, as in two-valued logic. This dichotomous assumption about cultural truth is a strong and potentially limiting assumption in the GCM. For example, questions similar to (1) seem appropriate for two logical values, whereas questions like (2) may be better represented by a continuous representation of degrees of cultural truth, e.g. a value in [0, 1], as in fuzzy logic (Zadeh, 1965).

The primary purpose of this paper is to introduce and compare a new CCT model for dichotomous items, the Latent Truth Model (LTM). The new model weakens the GCM assumption that cultural truth is two-valued by assuming that cultural truth is specified by parameters representing truth-values coded continuously in [0, 1]. The LTM shares some of the structure of a CCT model by Batchelder, Strashny, and Romney (2010) for cases where informant responses are in the interval [0, 1], as in judgments of the probability of events or the similarity between pairs of semantic terms. In this paper, we contrast the strong distinction of using CCT to infer truths as continuous values, as with the LTM, versus the traditionally-used, two-valued cultural truth specification in the GCM, first analytically, then with simulated data, and finally by applying both to published data.

The paper is divided into six sections. After the introductory section, Section 2 presents a generalized version of the GCM along with several new mathematical properties, and then Section 3 presents the new LTM model along with several of its mathematical properties as well. Section 4 compares the two models analytically, and discusses similarities and differences between the roles of the parameters specified in the two models. In this section, a statistic of the response data is provided that tends to distinguish the two models. Section 5 develops hierarchical Bayesian inference and posterior predictive checks for both models based on MCMC methods, and applies these developments to both simulated and published data. This paper is the first paper that presents hierarchical Bayesian inference for CCT models, and we hope that it sets the stage for others to adopt this approach. Finally, in Section 6 we discuss suggested conditions that need to be met for the appropriate use of each model.