Living Life to the Fullest: Long-term Relationships with Embodied Conversational Agents

By Justine Cassell, Northwestern University

An Embodied Conversational Agent (ECA) is a multimodal interface that adopts some of the properties of human face-to-face conversation, including producing and responding to verbal and non-verbal input and using conversational functions such as turn taking and feedback.  ECAs rely on the visual dimension of interacting with an animated character on a screen.  Generating conversational behavior for ECAs therefore depends on insights from graphics, vision, speech recognition and synthesis, artificial intelligence, human-computer interface design, and computational linguistics.  

My work in this interdisciplinary domain (a) highlights the relationship between discourse phenomena and non-verbal behaviors in conversations with computers and between humans mediated by computers, (b) demonstrates their rule-based generativity at a number of levels, and (c) evaluates their effect on human - computer interaction.  In this address I will talk about generation of language and graphics (nonverbal behaviors for animated agents) from both underlying concepts and typed text.  I will describe new work that extends notions of conversational interaction between computers and humans to include engaging in social chit-chat, establishing common ground, corroborating and criticizing.  And I will demonstrate implementations (and their evaluations) ranging from handheld devices to graphical online worlds, information kiosks and interactive learning systems for young children.

Anthropomorphic systems such as these embodied conversational agents have generated considerable controversy -- some go so far as to imagine nightmare scenarios in which sentient virtual humans take over thinking from humans.  In this talk I will no doubt simultaneously assuage some old fears, and evoke some new nightmares with a description of a new world where Embodied Conversational Agents become our closest confidants.