Cognitive-emotive Robotics for Supporting Autobiographic Storytelling

The aim of the project is the development of a virtual agent that interactively supports older persons in the narration of autobiographic stories, from listener feedback to raw transcription.

The first major research steps include the development of a virtual listener architecture with affective speech understanding and affective non-verbal and verbal feedback generation, both in real-time. In a second step, taking into consideration the results of cognitive narratology, the listener agent will incorporate a story process modeller and a dialog partner modeller, developed specifically for story generation. Functions should extend to context-sensitive verbal feedback, non-interrupting repetitions of parts of speech, clarifying questions, proposals for topics, redirecting the persons to their main storyline, ordering of main topics and sub-topics, etc. In addition to adapting these functions individually to a dynamic speaker (narrator) model, the virtual agent has to display coordinated action and expression in line with the story and the narrator. The current project covers the first phase of developing an active virtual listener and to test its effects on human narrators and the quality of their stories.

The project builds on the results of EU-sponsored projects in which OFAI acted as coordinator or partner, e.g. IRIS (Integrating Research in Interactive Storytelling), CYBEREMOTIONS (Collective Emotions in Cyberspace), SERA (Social Engagement with Robots and Agents), "Interactive Entertainment of Older Persons with Intelligent and Emotional Software-Agents" and "Self-Awareness and User-Awareness through Theory of Mind and Empathy", the latter two supported by the Austrian Federal Ministry for Transport, Innovation, and Technology.

Research staff

• Robert Trappl
• Sabine Payr
• Marcin Skowron
• Karin Vorsteher

Sponsor