Abstract
In two experiments, the author investigated how the correspondence between a verbal representation (a text describing data) and a simultaneously presented graphical representation (a graph showing data) affects comprehension of, and memory for, the results of empirical studies. Consistency of text and graph was varied by verbally describing observed differences between experimental conditions as small or large, and by using graphical depictions that made the effects appear to be small or large. In accord with the structure mapping hypothesis suggested by Schnotz and Bannert (2003), inconsistency of texts and graphs slowed down comprehension. Such inconsistency increased gaze durations on texts and graphs as well as the number of eye movements between text and graph. It also decreased confidence in a later memory task. Moreover, memory for the text was affected by the graph and memory for the graph was affected by the text, yielding distortions in cases of inconsistency.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
To avoid confusion, the two experiments reported here are consistently referred to as “experiments”, whereas the experiments described to the participants are called “studies” or “experimental studies”.
References
Cleveland, W. S. (1993). Visualizing data. Summit, NJ: Hobart.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
Glenberg, A. M., & Langston, W. E. (1992). Comprehension of illustrated text: Pictures help to build mental models. Journal of Memory and Language, 31, 129–151.
Godden, D. R., & Baddeley, A. D. (1975). Context-dependent memory in two natural environments: On land and under water. British Journal of Psychology, 66, 325–331.
Kosslyn, S. M. (1994). Elements of graph design. New York: Freeman.
Mayer, R. E. (1989). Systematic thinking fostered by illustrations in scientific text. Journal of Educational Psychology, 81, 240–246.
Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32, 1–19.
Mayer, R. E., & Gallini, J. K. (1990). When is an illustration worth ten thousand words? Journal of Educational Psychology, 82, 715–726.
Rinck, M. (2000). Situationsmodelle und das Verstehen von Erzähltexten: Befunde und Probleme. (Situation models and the comprehension of narrative text: Findings and problems.) Psychologische Rundschau, 51, 115–122.
Schnotz, W. (1993). On the relation between dual coding and mental models in graphics comprehension. Learning and Instruction, 3, 247–249.
Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13, 141–156.
Tufte, E. R. (1983). The visual display of quantitative information. Cheshire, CT: Graphics Press.
Wainer, H. (1984). How to display data badly. The American Statistician, 38, 137–147.
Williams , P., & Tarr , M. J. (no date). RSVP: Experimental control software for MacOS [Online]. Available: http://psych.umb.edu/rsvp/ [1998, October 27].
Acknowledgments
Preparation of this paper was supported by grant Ri 600/3-3 from the German Research Foundation (DFG) to Mike Rinck. I would like to thank Ariane Bürkner, Constanze Hesse, Anja Pongracz, and Alexander Varchmin for their help in preparing and conducting the experiments. I am also grateful to two anonymous reviewers for helpful comments on an earlier version of this chapter.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Sample Experimental Study Description, translated from German
-
1- Introduction:
-
This experiment investigated how memory depends on the context. A group of scuba divers learned lists of words. Half of them learned on land, and the other half learned under water. Afterwards, half of each group were asked to recall the words on land or under water. The experimenters counted how many words were recalled in each of the four possible combinations of learning environment (on land vs. under water) and recall environment (on land vs. under water). They expected that recall would be better when learning environment and recall environment are identical.
-
2a- Results Description Suggesting Large Effect:
-
Indeed, divers who both learned and recalled on land remembered a very high number of words (33 on average). Similarly, divers who learned and recalled under water, also recalled a very high number of words (31). However, if the environment changed, recall was much worse: Learning on land and recalling under water yielded a mean of only 27 words, and learning under water and recalling on land yielded only 28 words.
-
2b- Results Description Suggesting Small Effect:
-
However, divers who both learned and recalled on land remembered a medium number of words (33 on average). Similarly, divers who learned and recalled under water, also recalled a medium number of words (31). If the environment changed, recall was almost as good: Learning on land and recalling under water yielded a mean of 27 words, and learning under water and recalling on land even yielded 28 words.
Rights and permissions
Copyright information
© 2008 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Rinck, M. (2008). The Interaction of Verbal and Pictorial Information in Comprehension and Memory. In: Rouet, JF., Lowe, R., Schnotz, W. (eds) Understanding Multimedia Documents. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-73337-1_10
Download citation
DOI: https://doi.org/10.1007/978-0-387-73337-1_10
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-73336-4
Online ISBN: 978-0-387-73337-1
eBook Packages: Humanities, Social Sciences and LawEducation (R0)