Top

Gepubliceerd in:

25-04-2017 | Original Article

A new statistical model for analyzing rating scale data pertaining to word meaning

Auteurs: Felipe Munoz-Rubke, Karen Kafadar, Karin H. James

Gepubliceerd in: Psychological Research | Uitgave 4/2018

Abstract

The concrete-abstract categorization scheme has guided several research programs. A popular way to classify words into one of these categories is to calculate a word’s mean value in a Concreteness or Imageability rating scale. However, this procedure has several limitations. For instance, results can be highly distorted by outliers, ascribe differences among words when none may exist, and neglect rating trends in participants. We suggest using an alternative procedure to analyze rating scale data called median polish analysis (MPA). MPA is tolerant to outliers and accounts for information in multiple dimensions, including trends among participants. MPA performance can be readily evaluated using an effect size measure called analog R ² and be integrated with bootstrap 95% confidence intervals, which can prevent assigning inexistent differences among words. To compare these analysis procedures, we asked 80 participants to rate a set of nouns and verbs using four different rating scales: Action, Concreteness, Imageability, and Multisensory. We analyzed the data using both two-way and three-way MPA models. We also calculated 95% CIs for the two-way models. Categorizing words with the Action scale revealed a continuum of word meaning for both nouns and verbs. The remaining scales produced dichotomous or stratified results for nouns, and continuous results for verbs. While the sample mean analysis generated continua irrespective of the rating scale, MPA differentiated among dichotomies and continua. We conclude that MPA allowed us to better classify words by discarding outliers, focusing on main trends, and considering the differences in rating criteria among participants.

vorige artikel Sequential modulation of (bottom–up) response activation and inhibition in a response conflict task: a single-pulse transcranial magnetic stimulation study

volgende artikel False memory susceptibility in coma survivors with and without a near-death experience

Alleen toegankelijk voor geautoriseerde gebruikers

Allen, R., & Hulme, C. (2006). Speech and language processing mechanisms in verbal serial recall. Journal of Memory and Language, 55(1), 64–88. doi:10.1016/j.jml.2006.02.002.CrossRef

Altarriba, J., Bauer, L. M., & Benvenuto, C. (1999). Concreteness, context availability, and imageability ratings and word associations for abstract, concrete, and emotion words. Behavior Research Methods, Instruments, & Computers: A Journal of the Psychonomic Society Inc, 31(4), 578–602.CrossRef

Amsel, B. D., Urbach, T. P., & Kutas, M. (2012). Perceptual and motor attribute ratings for 559 object concepts. Behavior Research Methods, 44(4), 1028–1041. doi:10.3758/s13428-012-0215-z.PubMedPubMedCentralCrossRef

Balota, D. A., Yap, M. J., Cortese, M. J., Hutchison, K. A., Kessler, B., Loftis, B., & Treiman, R. (2007). The English Lexicon Project. Behavior Research Methods, 39(3), 445–459.PubMedCrossRef

Barsalou, L. W. (1999). Perceptual symbol systems. The Behavioral and Brain Sciences, 22(4), 577–609. (discussion 610–660).PubMed

Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59(1), 617–645. doi:10.1146/annurev.psych.59.103006.093639.PubMedCrossRef

Binder, J. R., Westbury, C. F., McKiernan, K. A., Possing, E. T., & Medler, D. A. (2005). Distinct brain systems for processing concrete and abstract concepts. Journal of Cognitive Neuroscience, 17(6), 905–917.PubMedCrossRef

Borghi, A. M., & Binkofski, F. (2014). Words as social tools: An embodied view on abstract concepts. New York: Springer.CrossRef

Borghi, A. M., Binkofski, F., Castelfranchi, C., Cimatti, F., Scorolli, C., & Tummolini, L. (2017). The challenge of abstract concepts. Psychological Bulletin, 143(3), 263–292. doi:10.1037/bul0000089.PubMedCrossRef

Borghi, A. M., Flumini, A., Cimatti, F., Marocco, D., & Scorolli, C. (2011). Manipulating objects and telling words: A study on concrete and abstract words acquisition. Frontiers in Psychology. doi:10.3389/fpsyg.2011.00015.

Borghi, A. M., Scorolli, C., Caligiore, D., Baldassarre, G., & Tummolini, L. (2013). The embodied mind extended: Using words as social tools. Frontiers in Psychology. doi:10.3389/fpsyg.2013.00214.PubMedPubMedCentral

Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46(3), 904–911. doi:10.3758/s13428-013-0403-5.PubMedCrossRef

Candès, E. J., Li, X., Ma, Y., & Wright, J. (2011). Robust principal component analysis? Journal of the ACM, 58(3), 1–37. doi:10.1145/1970392.1970395.CrossRef

Collins, A. M., & Loftus, E. F. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82(6), 407–428. doi:10.1037//0033-295X.82.6.407.CrossRef

Connell, L., & Lynott, D. (2012). Strength of perceptual experience predicts word processing performance better than concreteness or imageability. Cognition, 125(3), 452–465. doi:10.1016/j.cognition.2012.07.010.PubMedCrossRef

Crutch, S. J., Troche, J., Reilly, J., & Ridgway, G. R. (2013). Abstract conceptual feature ratings: The role of emotion, magnitude, and other cognitive domains in the organization of abstract conceptual knowledge. Frontiers in Human Neuroscience. doi:10.3389/fnhum.2013.00186.

Cumming, G. (2008). Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 3(4), 286–300. doi:10.1111/j.1745-6924.2008.00079.x.PubMedCrossRef

Cumming, G. (2014). The new statistics: Why and how. Psychological Science, 25(1), 7–29. doi:10.1177/0956797613504966.PubMedCrossRef

de Groot, A. M. (1989). Representational aspects of word imageability and word frequency as assessed through word association. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(5), 824–845. doi:10.1037/0278-7393.15.5.824.

Della Rosa, P. A., Catricalà, E., Vigliocco, G., & Cappa, S. F. (2010). Beyond the abstract—concrete dichotomy: Mode of acquisition, concreteness, imageability, familiarity, age of acquisition, context availability, and abstractness norms for a set of 417 Italian words. Behavior Research Methods, 42(4), 1042–1048. doi:10.3758/BRM.42.4.1042.PubMedCrossRef

Everitt, B., & Hothorn, T. (2011). An introduction to applied multivariate analysis with R. New York: Springer.CrossRef

Fiebach, C. J., & Friederici, A. D. (2004). Processing concrete words: fMRI evidence against a specific right-hemisphere involvement. Neuropsychologia, 42(1), 62–70.PubMedCrossRef

Fliessbach, K., Weis, S., Klaver, P., Elger, C. E., & Weber, B. (2006). The effect of word concreteness on recognition memory. NeuroImage, 32(3), 1413–1421. doi:10.1016/j.neuroimage.2006.06.007.PubMedCrossRef

Ghio, M., Vaghi, M. M. S., & Tettamanti, M. (2013). Fine-grained semantic categorization across the abstract and concrete domains. PLoS One, 8(6), e67090. doi:10.1371/journal.pone.0067090.PubMedPubMedCentralCrossRef

Giesbrecht, B., Camblin, C. C., & Swaab, T. Y. (2004). Separable effects of semantic priming and imageability on word processing in human cortex. Cerebral Cortex (New York, N.Y.: 1991), 14(5), 521–529. doi:10.1093/cercor/bhh014.CrossRef

Griffiths, T. L., & Steyvers, M. (2004). Finding scientific topics. Proceedings of the National Academy of Sciences, 101(Supplement 1), 5228–5235. doi:10.1073/pnas.0307752101.CrossRef

Hoaglin, D., Mosteller, F., & Tukey, J. W. (Eds.). (2000). Understanding robust and exploratory data analysis (Wiley classics library ed.). New York: Wiley.

Hoaglin, D., Mosteller, F., & Tukey, J. W. (Eds.). (2006). Exploring data tables, trends, and shapes. Hoboken: Wiley-Interscience.

Hoffman, P., & Lambon Ralph, M. A. (2013). Shapes, scents and sounds: Quantifying the full multi-sensory basis of conceptual knowledge. Neuropsychologia, 51(1), 14–25. doi:10.1016/j.neuropsychologia.2012.11.009.PubMedCrossRef

Hofmann, T. (1999). Probabilistic latent semantic indexing. In Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '99) (pp. 50–57). New York, NY, USA:ACM Press. doi:10.1145/312624.312649.

Hubert, M., & Debruyne, M. (2009). Breakdown value. Wiley Interdisciplinary Reviews: Computational Statistics, 1(3), 296–302. doi:10.1002/wics.34.CrossRef

Jamieson, S. (2004). Likert scales: How to (ab)use them. Medical Education, 38(12), 1217–1218. doi:10.1111/j.1365-2929.2004.02012.x.PubMedCrossRef

Jessen, F., Heun, R., Erb, M., Granath, D. O., Klose, U., Papassotiropoulos, A., & Grodd, W. (2000). The concreteness effect: Evidence for dual coding and context availability. Brain and Language, 74(1), 103–112. doi:10.1006/brln.2000.2340.PubMedCrossRef

Jones, M. N., & Mewhort, D. J. K. (2007). Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114(1), 1–37. doi:10.1037/0033-295X.114.1.1.PubMedCrossRef

Kiefer, M., & Barsalou, L. W. (2013). Grounding the human conceptual system in perception, action, and internal states. In W. Prinz, M. Beisert, & A. Herwig (Eds.), Action Science: Foundations of an Emerging Discipline (pp. 381–407). Cambridge, MA:MIT Press.

Kloke, J., & McKean, J. W. (2015). Nonparametric statistical methods using R. Boca Raton: CRC Press, Taylor & Francis.

Kousta, S.-T., Vigliocco, G., Vinson, D. P., Andrews, M., & Del Campo, E. (2011). The representation of abstract words: Why emotion matters. Journal of Experimental Psychology. General, 140(1), 14–34. doi:10.1037/a0021446.PubMedCrossRef

Kroll, J. F., & Merves, J. S. (1986). Lexical access for concrete and abstract words. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12(1), 92–107. doi:10.1037/0278-7393.12.1.92.

Landauer, T. K. (1999). Latent semantic analysis: A theory of the psychology of language and mind. Discourse Processes, 27(3), 303–310. doi:10.1080/01638539909545065.CrossRef

Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2), 211–240. doi:10.1037/0033-295X.104.2.211.CrossRef

Levelt, W. J., Roelofs, A., & Meyer, A. S. (1999). A theory of lexical access in speech production. The Behavioral and Brain Sciences, 22(1), 1–38. (discussion 38–75).PubMed

Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28(2), 203–208. doi:10.3758/BF03204766.CrossRef

Mahon, B. Z., & Caramazza, A. (2008). A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content. Journal of Physiology-Paris, 102(1–3), 59–70. doi:10.1016/j.jphysparis.2008.03.004.CrossRef

Nelson, D. L., & Schreiber, T. A. (1992). Word concreteness and word structure as independent determinants of recall. Journal of Memory and Language, 31(2), 237–260. doi:10.1016/0749-596X(92)90013-N.CrossRef

Paivio, A. (1986). Mental representations: A dual coding approach. Oxfort: Oxford University Press.

Paivio, A. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology/Revue Canadienne de Psychologie, 45(3), 255–287. doi:10.1037/h0084295.CrossRef

Paivio, A., Yuille, J. C., & Madigan, S. A. (1968). Concreteness, imagery, and meaningfulness values for 925 nouns. Journal of Experimental Psychology, 76(1, Pt.2), 1–25. doi:10.1037/h0025327.CrossRef

Papagno, C., Fogliata, A., Catricalà, E., & Miniussi, C. (2009). The lexical processing of abstract and concrete nouns. Brain Research, 1263, 78–86. doi:10.1016/j.brainres.2009.01.037.PubMedCrossRef

Papagno, C., Martello, G., & Mattavelli, G. (2013). The neural correlates of abstract and concrete words: Evidence from brain-damaged patients. Brain Sciences, 3(3), 1229–1243. doi:10.3390/brainsci3031229.PubMedPubMedCentralCrossRef

Peirce, J.W. (2007). PsychoPy–Psychophysics software in Python. Journal of Neuroscience Methods, 162(1-2), 8–13. doi:10.1016/j.jneumeth.2006.11.017.PubMedPubMedCentralCrossRef

R Core Team. (2016). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.

Recchia, G., & Jones, M. N. (2012). The semantic richness of abstract concepts. Frontiers in Human Neuroscience. doi:10.3389/fnhum.2012.00315.PubMedPubMedCentral

Rodríguez-Ferreiro, J., Gennari, S. P., Davies, R., & Cuetos, F. (2011). Neural correlates of abstract verb processing. Journal of Cognitive Neuroscience, 23(1), 106–118. doi:10.1162/jocn.2010.21414.PubMedCrossRef

Romani, C., Mcalpine, S., & Martin, R. C. (2008). Concreteness effects in different tasks: Implications for models of short-term memory. The Quarterly Journal of Experimental Psychology, 61(2), 292–323. doi:10.1080/17470210601147747.PubMedCrossRef

Sabsevitz, D. S., Medler, D. A., Seidenberg, M., & Binder, J. R. (2005). Modulation of the semantic system by word imageability. NeuroImage, 27(1), 188–200. doi:10.1016/j.neuroimage.2005.04.012.PubMedCrossRef

Schwanenflugel, P. J., Akin, C., & Luh, W.-M. (1992). Context availability and the recall of abstract and concrete words. Memory & Cognition, 20(1), 96–104. doi:10.3758/BF03208259.CrossRef

Schwanenflugel, P. J., & LaCount, K. L. (1988). Semantic relatedness and the scope of facilitation for upcoming words in sentences. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14(2), 344–354. doi:10.1037/0278-7393.14.2.344.

Schwanenflugel, P. J., & Shoben, E. J. (1983). Differential context effects in the comprehension of abstract and concrete verbal materials. Journal of Experimental Psychology: Learning, Memory, and Cognition, 9(1), 82–102. doi:10.1037/0278-7393.9.1.82.

Sidhu, D. M., Kwan, R., Pexman, P. M., & Siakaluk, P. D. (2014). Effects of relative embodiment in lexical and semantic processing of verbs. Acta Psychologica, 149, 32–39. doi:10.1016/j.actpsy.2014.02.009.PubMedCrossRef

Spence, I., & Lewandowsky, S. (1989). Robust multidimensional scaling. Psychometrika, 54(3), 501–513. doi:10.1007/BF02294632.CrossRef

Sullivan, G. M., & Artino, A. R. (2013). Analyzing and interpreting data from Likert-type scales. Journal of Graduate Medical Education, 5(4), 541–542. doi:10.4300/JGME-5-4-18.PubMedPubMedCentralCrossRef

Tillotson, S. M., Siakaluk, P. D., & Pexman, P. M. (2008). Body–object interaction ratings for 1,618 monosyllabic nouns. Behavior Research Methods, 40(4), 1075–1078. doi:10.3758/BRM.40.4.1075.PubMedCrossRef

Troche, J., Crutch, S., & Reilly, J. (2014). Clustering, hierarchical organization, and the topography of abstract and concrete nouns. Frontiers in Psychology. doi:10.3389/fpsyg.2014.00360.PubMedPubMedCentral

Tsai, P.-S., Yu, B. H.-Y., Lee, C.-Y., Tzeng, O. J.-L., Hung, D. L., & Wu, D. H. (2009). An event-related potential study of the concreteness effect between Chinese nouns and verbs. Brain Research, 1253, 149–160. doi:10.1016/j.brainres.2008.10.080.PubMedCrossRef

Tukey, J. W. (1977). Exploratory data analysis. Reading: Addison-Wesley Pub. Co.

Tyler, L. K., Russell, R., Fadili, J., & Moss, H. E. (2001). The neural representation of nouns and verbs: PET studies. Brain: A Journal of Neurology, 124, 1619–1634.CrossRef

Varela, F. J., Thompson, E., & Rosch, E. (2000). The embodied mind: Cognitive science and human experience (8th ed.). Cambridge, Mass.: MIT Press.

Venables, W. N., & Ripley, B. D. (2010). Modern applied statistics with S (4th ed.). New York: Springer.

Wickham, H. (2009). Ggplot2: Elegant graphics for data analysis. New York: Springer.CrossRef

Wiemer-Hastings, K., Krug, J. D., & Xu, X. (2001). Imagery, context availability, contextual constraint and abstractness. In J. D. Moore & K. Stenning (Eds.), Proceedings of the Twenty-third Annual Conference of the Cognitive Science Society (pp. 1106–1111). Mahwah: Lawrence Erlbaum Associates Inc.

Titel: A new statistical model for analyzing rating scale data pertaining to word meaning
Auteurs: Felipe Munoz-Rubke
Karen Kafadar
Karin H. James
Publicatiedatum: 25-04-2017
Uitgeverij: Springer Berlin Heidelberg
Gepubliceerd in: Psychological Research / Uitgave 4/2018
Print ISSN: 0340-0727
Elektronisch ISSN: 1430-2772
DOI: https://doi.org/10.1007/s00426-017-0864-8

Bohn Stafleu van Loghum

Deel dit onderdeel of sectie (kopieer de link)

Abstract

Log in om toegang te krijgen

Andere artikelen Uitgave 4/2018

On the time-course of automatic response activation in the Simon task

Perceptual impressions of causality are affected by common fate

Sequential modulation of (bottom–up) response activation and inhibition in a response conflict task: a single-pulse transcranial magnetic stimulation study

Contingency and contiguity of imitative behaviour affect social affiliation

Defining stimulus representation in stimulus–response associations formed on the basis of task execution and verbal codes

The effects of anxiety and situation-specific context on perceptual–motor skill: a multi-level investigation