CLASP
The Centre for Linguistic Theory and Studies in Probability

Towards Natural Dialogue with Robots

Robots can be more effective teammates with people if they can engage in natural language dialogue. In this talk, I will address one fundamental research problem to achieving this goal: understanding how people will talk to robots in collaborative tasks, and how robots could respond in natural language to maintain an effective dialogue that stays on track. The unique contribution of this research is the adoption of a multi-phased approach to building spoken dialogue systems that starts with exploratory data collection of human-robot dialogue with a human ¿wizard¿ standing in for the robot¿s language processing behind the scenes, and ends with training a dialogue system that automates away the wizard.With the ultimate goal of an autonomous conversational robot in mind, I will focus on the initial experiments that aim to collect computationally tractable human-robot dialogue without sacrificing naturalness. I will show how this approach can efficiently collect dialogue in the navigation domain, and in a form suitable for training a conversational robot. I will also present a novel annotation scheme for dialogue semantics and structure that captures the types of instructions that people gave to the robot, showing that over time these can change as people better assess the robot’s capabilities. Finally, I¿ll place this research effort in the broader context of enabling better teaming between people and robots.This is joint work with colleagues at ARL and at the USC Institute for Creative Technologies.