Ebook: From Communication to Presence

By nature, meaning design is not a homogeneous and univocal representation of reality but has an intrinsically composite character. Such hybrid theory of meaning entails that many aspects of meaning should be explained resorting to different layers and components of communication involving cognitive principles, knowledge factors, subjective experiences, and interactional principles, and no doubt much besides. Grice was essentially correct in thinking of meaning as a composite notion. The full meaning of an utterance may only be captured by considering different kinds of content: what is said, what is conventionally implicated, and what is presupposed.

Anolli, 2000

If there is one prerequisite that is critical for education, it is the need to communicate clearly and effectively. Communication is the core activity for an educator, conveying and sharing information from one person to another, from one organisation to another – or a combination of both. Professor Luigi Anolli has been one of the most distinguished leaders in the field of communication psychology in the past three decades through his contributions and his deep influence on many researchers and colleagues. Throughout his long career, he has made fundamental and profound contributions to various areas of psychology, including communication, emotion and culture psychology.

Luigi is one of the pioneers in the psychological study of miscommunication. Overcoming the traditional concept of miscommunication as a lack, fault and violation of rules, he has advanced modern communication psychology in a unique and far reaching manner. In this perspective, detailed in his Miscommunication as CHance Theory (MaCHT), communication sets up a unique and global category, which definitely includes miscommunication phenomena as important objects of investigation. Luigi also had a significant impact on defining and applying advanced techniques to understand and analyze a broad range of communicative and emotional phenomena – from Irony and Deception to Shame – throughout the physiological parameters, vocal nonverbal features, facial expression and posture emerging within a communicative interaction.

Luigi's work has been seminal in a broad spectrum of psychological research; not only has he settled a long list of critical questions but he has also opened new directions of research which have inspired students and colleagues throughout the world. His most recent contributions are related to the psychology of culture, described as the “natural” horizon of communication. Luigi's research as well as the one of his colleagues over the years have highlighted the importance of culture in understanding human behavior. Increasingly intrigued by culture, over a decade ago he started to review the cross-cultural literature in all areas of psychology. Soon he realized that culture played an important role for understanding communication, gradually recognizing its pervasive and profound influence on psychological processes in all areas of functioning.

The method, context, structure, language, knowledge and an understanding of the needs of the recipients to whom the information is being transmitted are vital in understanding the importance of communication in a culture. Without a proper understanding of the complex and dynamic process of meaning co-construction and sharing, the communication between different cultures will be very difficult.

Like many other colleagues, I have benefited from his presence and from his generous help. In the summer of 2004, Luigi Anolli joined my Faculty at the University of Milan-Bicocca where he founded the Centre for Studies in Communication Science (CESCOM).

The chapters in this book are authored by colleagues and friends of Luigi and the topics covered are closely related to Luigi's research. In fact, the contributions included in this festschrift encompass a series of topics in communication psychology to which he contributed directly, exhibited an abiding interest, and/or supported indirectly through his role as Director of the CESCOM.

I am very grateful to all the authors who have contributed their excellent theoretical and research chapters to this book. At this occasion, I would like to express my deepest friendship and gratitude to Luigi Anolli and to wish him many more happy and productive years ahead.

Prof. Susanna Mantovani, Dean of the Science Education Faculty, University of Milan-Bicocca, Milan, Italy

The convergence between telecommunication, virtual reality and artificial intelligence technologies resulted in a dramatical increase and modification of the opportunities to experience the physical and social world. Their diffusion and integration into multi-user and multi-agent virtual worlds highlighted the relevance of addressing from a common psychological perspective the domain of communication and the domain of presence. New theoretical and practical questions are emerging, in the double intent to explain phenomena at the interplay between mind and technology and to design effective technological applications. This chapter has the goal to start an exploration of the links and reciprocal contributions between communication and presence, analyzed at theoretical, methodological and applicative level.

Presence is widely accepted as the key concept to be considered in any research involving human interaction with Virtual Reality (VR). Since its original description, the concept of presence has developed over the past decade to be considered by many researchers as the essence of any experience in a virtual environment.

The VR generating systems comprise two main parts: a technological component and a psychological experience. The different relevance given to them produced two different but coexisting visions of presence: the rationalist and the psychological/ecological points of view. The rationalist point of view considers a VR system as a collection of specific machines with the necessity of the inclusion of the concept of presence. The researchers agreeing with this approach describe the sense of presence as a function of the experience of a given medium (Media Presence). The main result of this approach is the definition of presence as the perceptual illusion of non-mediation produced by means of the disappearance of the medium from the conscious attention of the subject. At the other extreme, there is the psychological or ecological perspective (Inner Presence). Specifically, this perspective considers presence as a neuropsychological phenomenon, evolved from the interplay of our biological and cultural inheritance, whose goal is the control of the human activity.

Given its key role and the rate at which new approaches to understanding and examining presence are appearing, this chapter draws together current research on presence to provide an up to date overview of the most widely accepted approaches to its understanding and measurement.

In this chapter we will discuss the concepts of “presence” (Inner Presence) and “social presence” (Co-presence) within a cognitive and ecological perspective. Specifically, we claim that the concepts of “presence” and “social presence” are the possible links between self, action, communication and culture. In the first section we will provide a capsule view of Heidegger's work by examining the two main features of the Heideggerian concept of “being”: spatiality and “being with”. We argue that different visions from social and cognitive sciences – Situated Cognition, Embodied Cognition, Enactive Approach, Situated Simulation, Covert Imitation - and discoveries from neuroscience – Mirror and Canonical Neurons - have many contact points with this view. In particular, these data suggest that our conceptual system dynamically produces contextualized representations (simulations) that support grounded action in different situations. This is allowed by a common coding – the motor code – shared by perception, action and concepts. This common coding also allows the subject for natively recognizing actions done by other selves within the phenomenological contents. In this picture we argue that the role of presence and social presence is to allow the process of self-identification through the separation between “self” and “other,” and between “internal” and “external”. Finally, implications of this position for communication and media studies are discussed by way of conclusion.

We claim that presence is elicited most strongly when information is presented as an inhabitable, external world. Technical developments that permit this, such as the creation of interactive, immersive virtual environments herald a profound change in how people relate to sources of information, and how they communicate. This change has psychological, social and cultural effects. It has been claimed that in many ways, our relationship to information becomes that of our ancestral, pre-literate relationship to the physical world. By this view, we are heading for a post-literate future of body-based communication. But this view is too simple, since information must serve a variety of purposes, and how much presence is desirable in a communicative situation depends on many factors, including the communication devices available, the intended use and the context of use. In addition, differences between individuals, such as personality, as well as physical and psychological state, will affect how readily presence is invoked and also its impact on the individual concerned. In this chapter, we expand on the general notion of presence as a dimension of communication, and how this perspective can inform an understanding of designed variations in presence as a function of use, context, and individual psychological factors.

Advances in mediacommunication technology have opened up new avenues for understanding consciousness through observation of behaviour in virtual environments. A convergence of progress in cognitive neuroscience and computer science should consider the powerful role of conscious and unconscious states as an interface between self and virtual worlds. In this chapter, we review the premise of presence as a dimension of consciousness from both a phenomenological and neuroscientific perspective. Working from a model in which dreaming consciousness is considered the most archetypal form of media technology, dreams are discussed as a useful metaphor for virtual reality. We argue that presence can be equally compelling whether experienced via self-generated simulation during the process of dreaming, or through an externally generated media simulation. Attempts to use media technology in a therapeutic context need to consider clinical aspects of mechanisms involved in both normal and clinical/pathological aspects of consciousness. A speculative therapeutic approach, “dream simulation therapy”, is discussed as a future possible area of study. Dreaming consciousness reminds us that the key factor in approaching an ultimate technology-mediated presence experience is the sum rather than its parts: a subjective/affective state of being.

The highly demanding study of meaning, intention, and communication including miscommunication, in human interaction seems to call for the development of powerful new approaches and in that context the astonishing raw power of modern computers may eventually be harnessed, given that adequate models, methods, algorithms and software be developed and made available. In this context, a proposed data structure, pattern definitions, algorithms, and a new statistical validation test are proposed. New additions are introduced to this theoretical/methodological system (called t-system) including special definitions of well known phenomena such as bursts and cyclical occurrence as well as of more novel concepts called “t-blocks”, “t-metronomes” and “ghost cycles”. A method is introduced to deal with the estimation of a priori probability (or statistical significance) of individual patterns without consideration of the arbitrary binary trees used for their detection and in this context “t-templates” and their matching are introduced. Statistical validation through shuffling of data is compared with a suggested method called (random series) rotation (t-rotation) and results obtained with each are compared for both human and neuronal interactions. It is pointed out that brain behavior as observed with brain scanners does not offer direct insight into meaning and intentions, but essentially means more behavior to observe and more patterns to be detected, while limitations in social neuroscience seem to repeat to those of earlier human interaction studies and also due to technical difficulties. Finally some thoughts and questions are put forward concerning possible relations between on one hand hidden patterns and symmetry in interactions and on the other hand meaning, intentions, communication and miscommunication in highly patterned human interactions as well as about the possible need for new and specialized mathematics for the study of these phenomena.

The most recent research about both human-human conversational interaction and human-computer agents conversational interaction is marked by a multimodal perspective. On the one hand this approach underlines the co-occurrence and synergy between different languages and channels, on the other hand it highlights the need for joined and coordinated action between various subjects (attuning and mutual tuning in). In a similar way recent research on human computer interaction points out the need to consider the vocal interaction in a multicomponential perspective, both as a multilayer phenomenon in itself and as one component in wider interactive patterns. As the communicative action is seen with its features of comprehensiveness and multicomponentiality, so the vocal act needs to be seen as a complex event. Research on models aimed at new interfaces analysis, outline the way beyond the distinction traceable in the majority of studies, where conversational action is split up into its factors and analysis focuses on the factors one by one: conversation analysis or content analysis or suprasegmental analysis. The purpose of this chapter is to offer a contribution to the creation of an analysis model that allows for the complexity of the vocal act and for its being-in-context in the interactive flow, so applying the Embodied Conversational Agents (ECAs) qualifying multicomponential focus to the vocal act. Two levels of analysis so emerge: a vertical, morphological analysis, and a horizontal, sequential analysis.

Two kinds of vocal interaction are here examined according to the proposed model, a human face-to-face and an interaction between an ECA and its user.

This chapter is an overview of research on emotion in negotiation that integrates cognitive, affective, and cultural aspects of the field. We address the following issues: (1) the effects of mood and emotion on negotiator cognition and performance and the potential of emotion as a negotiation strategy; (2) individual differences in emotional expression and individual traits, such as self-monitoring and emotional intelligence, that impact on the use of strategic emotion; and (3) cultural influences on negotiation and on emotional experience and expression.

There is a long-standing interest in hidden temporal patterns in behavior. The current chapter discusses the idea that face-to-face interaction can be construed as having a definite organization or structure, just as language is understood in terms of its grammar. The participant has, within that organization, options he can exercise, including the option of violating aspects of the organization. Numerous studies, using the T-pattern detection algorithm, have demonstrated that the organization of behavior is influenced by situation, personality and culture. Strong relationships have been found between the structure of verbal and non-verbal communication and cognition and social adaptation. Little research exists though on the relation between real-time behavior organization and self-esteem and personality. An earlier study suggests a strong relationship between level of subject's self-esteem and number of real-time behavioral patterns produced in dyadic interaction situations. Significant differences have also been found in real-time behavioral patterns produced in dyadic interactions between subjects who considered themselves to be friends versus those who were strangers. It is unknown whether such behavioral analysis would reveal a difference in real-time patterns produced by persons with different scores on the Eysenck Personality Questionnaire. These ideas have been tested by analyzing twenty-four dyadic interactions between male students. A special software, THEME, was used to detect real-time patterns in real-time behavior records. Results indicate that these interactions are highly synchronized and structured. Strong correlation was found between subjects' self-esteem and complexity and frequency of behavioral patterns detected. Positive correlation was also found between subject's personality and complexity and frequency of patterns. Certain pattern types were found exclusively to be produced by extraverts and other by introverts. High and low self-esteem subjects' were also found to produce different types of behavioral patterns.

Human communication is a highly complex phenomenon that can be approached from numerous theoretical perspectives of varying nature. It is a good example of how traditional metatheoretical, epistemological and methodological controversies can be channeled through a body of knowledge whose aim is the rational and systematic search for different approaches, interconnections and complementary features.

The extraordinarily wide range of aspects to be considered and the experiential richness that goes hand in hand with every communicative episode make it necessary to choose observational methodology, capable of being both flexible and objective.

In the first stage of the process involving observational methodology, qualitative methodology is preferred for the study of communication given the wide range of options it provides in terms of data collection, but it's high relevant the characterization and application of a quantitative approach in the second stage of an observation of communicative behaviour. Current advances involving the use of sophisticated methodological resources, which enable much greater rigor through the process.

Given the growing interest in developing embodied virtual agents with multimodal communication and emotional expression abilities, the issue of user's involvement is a relevant topic to take into account in determining how to assess and interpret the quality of user-agent affective interaction. Main goal of this paper is the definition of a methodology for the analysis of user-agent interaction synchrony considered as an index of user's involvement. The proposed approach is based on recent advances in communication psychology, which on the one hand show the importance of considering the hidden temporal organization underlying communicative interaction and on the other hand provide a specific methodology for the structural analysis of the interactive flow (analysis of intra- and inter-individual multimodal behavioral patterns through Theme software). From a theoretical point of view, the crucial assumption is that the more synchronic interactions are, the more pleasant and fulfilling they are experienced, and consequently more related to positively valenced emotional states. Our main objective is to tune and to test this methodology (typically used in analyzing human-human communication exchanges) within user-agent interaction, in order to detect interactive temporal patterns of actions in affective interactions with virtual agents. This approach has been developed within the European project MYSELF, where we are a preliminary evaluation study of an interactive pedagogical agent by combining self-report measures, physiological measures and multimodal behavioral patterns approach is being carried out.

Since their emergence in the 1960s, computer games have developed into a central part of popular culture. An ever-increasing number of players plays games using their computers. One of the most successful forms of computer games is the phenomenon of multiplayer games, i.e. computer games that more than one player can participate in. In these games, various interaction and communication processes take place between the players as well as between the players and the virtual game spaces that these games provide. This chapter attempts to describe multiplayer games as a form of computer-mediated communication (CMC). This mode of communication has often been described as lacking certain social cues that a face-to-face situation provides. However, to understand communication and interaction processes, one needs to understand the situation in which these processes take place. The situation in which multiplayer games take place makes a large amount of cooperation and task-oriented interaction between the players necessary. This chapter attempts to examine communication processes in multiplayer first-person-shooter (FPS) games as determined by the gaming situation in as well as the social context of these games, emphasizing that communication in these games is successful despite the constraints it has in common with other forms of CMC.

Virtual reality (VR) driving simulators may be used as an aid to traditional cognitive-behavioral therapy in the treatment of a variety of driving-related disorders. In recent years there has been a heightened interest among researchers and clinicians in using VR technology to address a wide range of driving-related issues. Clinical applications include specific driving phobias, driving phobias related to panic and agoraphobia, and posttraumatic stress disorder (PTSD) as a result of motor vehicle accidents. Other areas of interest include neurorehabilitation for individuals who have sustained various brain injuries, examining the impact of pharmaceuticals while driving, and assessing and predicting driving abilities among teenage and elderly populations. The VR world elicits real reactions that can be modified through therapy to help people overcome disorders and traumas such as these. As with any type of treatment, some limitations exist. However, results thus far have been promising and directions for future research are discussed.

A learning environment for the training of communicative competence has to consider the complexity of human experience, since it requires a number of cues that are managed hic et nunc in the flow of communicative exchange. Therefore, communicative competence has been traditionally considered as a typical face-to-face learning topic. So far, few opportunities exist to learn by experience in an e-learning environment that can combine user's practising and experiencing with an adequate scaffolding structure, giving the learner both the opportunity to fail and the opportunity to give sense to the perspectives selected. Recent work on computer-based interactive simulations and autonomous agents is offering new opportunities for the training of communicative competence in different contexts. Simulation creates a unique environment for developing and executing communication skills. Moreover, the communicative interaction can be developed and enhanced in a realistic, but non-threatening situation. The present chapter aims at analysing how communication skills should be learned through computer-based interactive simulations. First, a definition of communication skills will be indicated considering their involvement in tackling communicative exchanges effectively. Second, an architecture for building interactive simulations will be proposed. In particular, a road map for building e-learning simulations specifically targeted at the training of communication skills will be sketched out, focusing on the development of a narrative structure that should adequately reduplicate the flow of the communicative interaction.