Project description

The project focused on the cognitive processes involved when learning with text and pictures. It was investigated whether text and pictures are mentally integrated with each other. Since text-picture integration is the precondition for successful learning, the results of this project are central to application fields where text and pictures are used as learning materials, for example in schools or universities.

The theoretical starting point of this project was the Cognitive Theory of Multimedia Learning by Richard Mayer (2009). The theory assumes that text and pictures are integrated with each other and that the integration process plays a central role for learning. The aim of the project was to shed light on the integration process.

Therefore, a paradigm was used where conflicting information is presented to the learner. It was assumed that if learners integrate text and pictures with each other, they have to become aware of text and picture elements corresponding to one another. This search for correspondences should occur during the learning process. If this assumption is true, learners should also realize conflicts between text and pictures. This research question was addressed in several studies using eye tracking. Gaze behavior of participants when processing a conflicting text-picture presentation was analyzed compared to a non-conflicting text-picture presentation. Thereby, static images were combined with written texts, as well as dynamic images with spoken texts. Processing of conflicting text-picture information was characterized by longer (written) text and picture fixation times and more transitions between (written) text and pictures. These findings indicate that text picture integration occurs early in the learning process.

Furthermore, it was investigated whether learners integrate also sequential information with each other. Here, contradictory information was presented successively. Results showed that learners realized the conflict and processed the information more intensively. These findings indicate that information presented simultaneously as well as sequentially is integrated into a mental model.

As information integration into a mental model is assumed to be helpful for learning, text-picture information should be supported especially within schools or universities.

Kooperationen

Dr. Martin Merkt, Deutsches Institut für Erwachsenenbildung