Multimodal Interaction Lab
Hypermedia-based learning environments are characterized by a non-linear connection of information nodes containing multimedia materials. These hyperlinked environments can be explored by users in an interactive and self-directed way.
The lab investigates three types of hypermedia environments: First, the World Wide Web (WWW) as a global network of information nodes of diverse origin and quality that can be explored by means of web browsers and search engines. Here, our research mainly addresses processes of evaluating and integrating different sources of information during web search. Second, instructional hypermedia environments that are designed to support learners of different needs. In particular, we are interested in how innovative technologies can be used to adapt these environments to learners’ prerequisites and to their learning progress. Third, we analyze the potentials of novel hypermedia environments that are based on interactive displays (e.g., multitouch tables, smartphones, or tablets). Here, we focus on the direct and intuitive manipulation of information by means of multitouch interaction and on the use of research paradigms from the field of embodied cognition for studying processes of knowledge acquisition.
With regard to all abovementioned types of hypermedia environments, we focus on two pivotal issues: How to support processes of comprehension by means of hypermedia and how to avoid potential cognitive overload due to the large amounts of information, presentation modes, and interaction requirements provided to users. We assume that a deeper understanding of hyperlinked information can be best supported when hypermedia design features are matched to users' particular prerequisites. Accordingly, the lab comprises two lines of research, addressing on the one hand the design and affordances of hyperlinked information environments, and on the other hand user prerequisites and cognitive resources.
Design and affordances of hyperlinked information environments
This line of research investigates how design features of hypermedia environments influence the resulting affordances with regard to user behavior and cognitive processing. For instance, we study which features of web resources and search interfaces stimulate users to elaborate on the origin and quality of web information. Moreover, we analyze which affordances for information processing are provided when embedding static and dynamic visualizations within hypermedia environments. A third core area is related to informal learning contexts like museums and addresses the design of intuitive hypermedia environments based on interactive displays. Here, we focus on the issue of how a direct haptic interaction with representations by means of gestures (“embodied interaction”) can not only improve an intuitive usability but also facilitate the acquisition of conceptual structures.
User prerequisites and cognitive resources
This line of research investigates important cognitive prerequisites for successful hypermedia exploration. One focus of the lab is on optimizing the management of limited cognitive resources like working memory capacity. For instance, we study how neural signatures of working-memory processes can be used to measure different aspects of working-memory load during learning in real time. The long-term goal of this work is to develop innovative and adaptive hypermedia environments, which allow reducing the amount of cognitive overload, particularly for users with unfavorable learning prerequisites. A second – and complementary – focus is on developing hypermedia environments for learners with exceptionally favorable learning prerequisites. For instance, we develop and optimize hypermedia-based iPad environments that address the specific learner prerequisites of gifted primary school children to stimulate discovery learning, e.g., in domains such as understanding biodiversity or musical structures.