Project description

The project focuses on the cognitive processes involved when learning with text and pictures. It is investigated whether text and pictures are mentally integrated with each other. Since text-picture integration is the precondition for successful learning, the results of this project are central to application fields where text and pictures are used as learning materials (e.g. school or university).

The theoretical starting point of this project is the Cognitive Theory of Multimedia Learning by Richard Mayer (2009). The theory assumes that text and pictures are integrated with each other and that the integration process plays a central role for learning. The aim of the project is to shed light on the integration process.

Therefore, a paradigm is used where conflicting information is presented to the learner. It is assumed that if learners integrate text and pictures with each other, they have to become aware of text and picture elements corresponding to one another. This search for correspondences should occur during the learning process. If this assumption is true, learners should also realize conflicts between text and pictures. This research question was addressed using eye tracking. Gaze behavior of participants when processing a conflicting text-picture presentation was analyzed compared to a non-conflicting text-picture presentation. Processing of conflicting text-picture information was characterized by longer text and picture fixation times and more transitions between text and pictures. These findings indicate that text picture integration occurs early in the learning process. Furthermore, it was investigated whether learners integrate also sequential information with each other. Here, contradictory information was presented successively. Results show that learners realized the conflict and processed the information more intensively. These findings indicate that information presented simultaneously as well as sequentially is integrated into a mental model.

As information integration into a mental model is assumed to be helpful for learning, text-picture information should be supported especially within schools or universities.

Publications

Schüler, A. (in press). The integration of information in a digital, multi-modal learning environment. Learning and Instruction. https://dx.doi.org/10.1016/j.learninstruc.2017.12.005

Schüler, A. (2017). Investigating gaze behavior during processing of inconsistent text-picture information: Evidence for text-picture integration. Learning and Instruction, 49, 218-231. https://dx.doi.org/10.1016/j.learninstruc.2017.03.001

Scheiter, K., Eitel, A., & Schüler, A. (2016). Lernen mit Texten und Bildern: Die frühzeitige wechselseitige Beeinflussung kognitiver Prozesse bei der Konstruktion eines integrierten mentalen Modells. Psychologische Rundschau, 2, 87-93.