mobile icon

The influence of hand proximity on cognitive and emotional information processing with multi-touch interfaces

Working groupMultimodal Interaction Lab
Duration01/2013 - open
FundingBudget resources
Project description

The use and application of interactive multi-touch displays increase continuously. Thus, touch-based user interfaces like multi-touch tables, tablets or smartphones are used in many public sector institutions and private households. These interfaces enable users to directly manipulate external representations using their hands without the requirement of an indirect interaction device (e.g., mouse). The question arises how touch-based interactions should be implemented to facilitate the processing of information.

Figure hand proximity
Example for hand proximity (presented in orange) while using interactive multi-touch interfaces

Recently, studies with simple reaction time tasks have shown that specific stimulus categories (e.g., pictures) are better processed in hand proximity than others. For example it has been shown that visuospatial processing (e.g., processing of pictures) is enhanced near the hands (Reed, Grubb, & Steele, 2006), whereas semantic processing (e.g., processing of sentences) might be impaired by hand proximity (Davoli, Du, Montana, & Garverick, 2010). Within this project, these investigations were expanded examining the influence of hand proximity on visuospatial and verbal learning contents, on cognitive control of different age groups and on emotion regulation. Results indicate that visuospatial information from pictures was better remembered when it was processed near the hands, whereas for verbal information from texts there was no such difference (Brucker, Ehrmann, & Gerjets, 2016). Furthermore, emotion regulation was enhanced when positive contents were presented near the hands compared to positive contents presented farther away from the hands (Ruiz Fernández, Lachmair, Rahona, & Gerjets, 2016). Prospective studies will examine the generalizability of these results to further application areas.

  • Brucker, B., Brömme, R., Ehrmann, A., Edelmann, J., & Gerjets, P. (2021). Touching digital objects directly on multi-touch devices fosters learning about visual contents. Computers in Human Behavior, 119, Article 106708.
  • Brömme, R., Gottschling, S., Brucker, B., & Gerjets, P. (2017, March). Upside down: hand proximity fosters attentional processing but not cognitive control in a visuospatial task-switching paradigm. 59. Conference of Experimental Psychologists (TeaP). Dresden. [Talk]
  • Ruiz Fernández, S., Lachmair, M., Rahona, J. J., & Gerjets, P. (2016, March). Grasping feelings: Proximity of hands to positive pictures boosts mood recovery. 58. Tagung experimentell arbeitender Psychologen (TeaP). Heidelberg. [Talk]