This volume brings together the invited papers and selected participants' contributions presented at the International NATO-ASI Summer School on “Fundamentals of Verbal and Nonverbal Communication and the Biometrical Issue”, held in Vietri sul Mare, Italy, September 2–12, 2006.

The School was jointly organized by the Faculty of Science and the Faculty of Psychology of the SECOND UNIVERSITY OF NAPLESCasertaItalythe INTERNATIONAL INSTITUTE for ADVANCED SCIENTIFIC STUDIES “Eduardo R. Caianiello” (IIASS), Vietri sul Mare, Italy, the ETTORE MAJORANA FOUNDATION and CENTRE FOR SCIENTIFIC CULTURE (EMFCSC), Erice, Italy, and the Department of Physics, UNIVERSITY OF SALERNO, Italy. The School was a NATO event, and although it was mainly sponsored by the NATO Programme SECURITY THROUGH SCIENCE, it also received contributions from the INTERNATIONAL SPEECH COMMUNICATION SOCIETY (ISCA) and the INTERNATIONAL SOCIETY OF PHONETIC SCIENCES (ISPhS), as well as from the abovementioned organizing Institutions.

The main theme of the school was the fundamental features of verbal and nonverbal communication and their relationships with the identification of a person, his/her socio-cultural background and personal traits. The problem of understanding human behaviour in terms of personal traits, and the possibility of an algorithmic implementation that exploits personal traits to identify a person unambiguously, are among the great challenges of modern science and technology. On the one hand, there is the theoretical question of what makes each individual unique among all others that share similar traits, and what makes a culture unique among various cultures. On the other hand, there is the technological need to be able to protect people from individual disturbance and dangerous behaviour that could damage an entire community.

As regards to the problem of understanding human behaviour, one of the most interesting research areas is that related to human interaction and face-to-face communication. It is in this context that knowledge is shared and personal traits acquire their significance. In the past decade, a number of different research communities within the psychological and computational sciences have tried to characterize human behaviour in face-to-face communication through several features that describe relationships between facial expressions and prosodic/voice quality; differences between formal and informal communication modes; cultural differences and individual and socio-cultural variations; stable personality traits and their degree of expressiveness and emphasis, as well as the individuation of emotional and psychological states of the interlocutors. There has been substantial progress in these different communities and surprising convergence, and the growing interest of researchers in understanding the essential unity of the field makes the current intellectual climate an ideal one for organizing a Summer School devoted to the study of verbal and nonverbal aspects of face-to-face communication and of how they could be used to characterize individual behaviour.

The basic intention of the event was to provide broad coverage of the major developments in the area of biometrics as well as the recent research on verbal and nonverbal features exploited in face-to-face communication. The focus of the lectures and the discussions was primarily on deepening the connections between the emerging field of technology devoted to the identification of individuals using biological traits (such as voice, face, fingerprints, and iris recognition) and the fundamentals of verbal and nonverbal communication which includes facial expressions, tones of voice, gestures, eye contact, spatial arrangements, patterns of touch, expressive movement, cultural differences, and other “nonverbal” acts. The main objective of the organizers was to bring together some of the leading experts from both fields and, by presenting recent advances in the two disciplines, provide an opportunity for cross-fertilization of ideas and for mapping out territory for future research and possible cooperation. The lectures and discussions clearly revealed that research in biometrics could profit from a deeper connection with the field of verbal and nonverbal communication, where personal traits are analyzed in the context of human interaction and the communication Gestalt.

Several key aspects were considered, such as the integration of algorithms and procedures for the recognition of emotional states, gesture, speech and facial expressions, in anticipation of the implementation of other useful applications such as intelligent avatars and interactive dialog systems.

Features of verbal and nonverbal communication were studied in detail and their links to mathematics and statistics were made clear with the aim of identifying useful models for biometric applications.

Recent advances in biometrics application were presented, and the features they exploit were described. Students departed from the Summer School having gained not only a detailed understanding of many of the recent tools and algorithms utilized in biometrics but also an appreciation for the importance of a multidisciplinary approach to the problem through the analysis and study of face-to-face interactions.

The contributors to this volume are leading authorities in their respective fields. We are grateful to them for accepting our invitation and making the school such a worthwhile event through their participation.

The contributions in the book are divided into four sections according to a thematic classification, even though all the sections are closely connected and all provide fundamental insights for cross-fertilization of different disciplines.

The first section, GESTURES and NONVERBAL BEHAVIOUR, deals with the theoretical and practical issue of assigning a role to gestural expressions in the realization of communicative actions. It includes the contributions of some leading experts in gestures such as Adam KENDON and David MCNEILL, the papers of Stefanie SHATTUCK-HUFNAGEL et al., Anna ESPOSITO and Maria MARINARO, Nicla ROSSINI, Anna ESPOSITO et al., and Sari KARJALAINEN on the search for relationships between gestures and speech, as well as two research works on the importance of verbal and nonverbal features for successful communication, discussed by Maja BRATANIĆ, and Krzysztof KORŻYK.

The second section, NONVERBAL SPEECH, is devoted to underlining the importance of prosody, intonation, and nonverbal speech utterances in conveying key aspects of a message in face-to-face interactions. It includes the contributions of key experts in the field, such as Nick CAMPBELL, Eric KELLER, and Ruth BAHR as research papers, and related applications proposed by Klara VICSI, Ioana VASILESCU and Martine ADDA-DECKER, Vojtěch STEJSKAL et al., Ke LI et al., Elina SAVINO, and Iker LUENGO et al. Further, this section includes algorithms for textual fingerprints and for web-based text retrieval by Carl VOGEL, Fausto IACCHELLI et al., and Stefano SQUARTINI et al.

The third section, FACIAL EXPRESSIONS, introduces the concept of facial signs in communication. It also reports on advanced applications for the recognition of facial expressions and facial emotional states. The section starts with a theoretical paper by Neda PINTARIĆ on pragmemes and pragmaphrasemes and goes on to suggest advanced techniques and algorithms for the recognition of faces and facial expressions in the papers by Praveen KAKUMANU and Nikolaos BOURBAKIS, Paola CAMPADELLI et al., Marcos FAUNDEZ-ZANUY, and Marco GRASSI.

The fourth section, CONVERSATIONAL AGENTS, deals with psychological, pedagogical and technological issues related to the implementation of intelligent avatars and interactive dialog systems that exploit verbal and nonverbal communication features. The section contains outstanding papers by Dominic MASSARO, Gerard BAILLY et al., David HOUSE and Björn GRANSTRÖM, Christopher PETERS et al., Anton NIJHOLT et al., and Bui TRUNG et al.

The editors would like to thank the NATO Programme SECURITY THROUGH SCIENCE for its support in the realization and publication of this edition, and in particular the NATO Representative Professor Ragnhild SOHLBERG for taking part in the meeting and for her enthusiasm and appreciation for the proposed lectures. Our deep gratitude goes to Professors Isabel TRANCOSO and Jean-Francois BONASTRE of ISCA, for making it possible for several students to participate through support from ISCA. Great appreciation goes to the dean of the Faculty of Science at the Second University of Naples, Professor Nicola MELONE, for his interest and support for the event, and to Professor Luigi Maria RICCIARDI, Chairman of the Graduate Program on Computational and Information Science, University of Naples Federico II, for his involvement and encouragement. The help of Professors Alida LABELLA and Giovanna NIGRO, respectively dean of the Faculty and director of the Department of Psychology at the Second University of Naples, is also acknowledged with gratitude.

Special appreciation goes to Michele DONNARUMMA, Antonio NATALE, and Tina Marcella NAPPI of IIASS, whose help in the organization of the School was invaluable.

Finally, we are most grateful to all the contributors to this volume and all the participants in the 2006 Vietri Summer School for their cooperation, interest, enthusiasm and lively interactions, making it not only a scientifically stimulating gathering but also a memorable personal experience.

This book is dedicated to those who struggle for peace and love, since peace and love are what keep us persevering in our research work.

The EDITORS: Anna ESPOSITO, Maja BRATANIĆ, Eric KELLER, Maria MARINARO