Projects‎ > ‎

Processing Social Signals in Human-Computer Interaction

Social signals are the expression of one’s attitude towards social interaction and social situations, conveyed through a set of nonverbal and behavioral cues.

The overall goal of this project is to learn in a context of human-computer interaction which social signals express the user’s level of experience, the quality of the interaction and interaction incidents, with the focus on computationally easy to detect signals. 

We are concerned with the specific context of interaction with public space, utility machines, such as self-service machines in car parks, supermarkets, theatres, bus and train stations, open space photocopiers, vending machines, among others.

This study follows the Social Signal Processing (SSP) methodology [Vinciarelli et al., 2008; Pantic et al., 2011], which proposes the analysis of human nonlinguistic behavior to make inferences on social relations, attitudes and roles, and to predict the behavioral outcomes of a particular social situation.

The overall goal of this project is to develop a model leading to the implementation of a measurement device capable of monitoring those social signals and classifying user confidence/level of experience, quality of interaction and the existence of critical incidents. The ability to detect social cues and correctly interpret them might then lead to systems that are better designed to provide more effective responses to the users’ doubts, difficulties and individual pace and thus generate a more natural and productive interaction.


João Pedro Ferreira 

Marta Noronha e Sousa 

Nuno Branco

Nuno Otero

Nelson Zagalo 

Manuel João Ferreira 

Pedro Branco

A project supported by: