Grounded Language Learning through Interaction

• Event: Seminar
• Lecturer: Bill Noble and Nikolai Ilinykh from University of Gothenburg
• Date: 04 May 2022
• Duration: 2 hours
• Venue: Gothenburg and Online

Abstract:

“One proposal for perceptually grounded representations of meaning is to identify perceptual meaning with perceptual classification. However, perceptual meaning need not be grounded directly in first-hand experience. If someone who doesn’t know the meaning of raven is told that a raven is a large black corvid, they may get some idea of how to identify ravens, even if they have never seen one. In this scenario, an agent grounds the novel concept through linguistic communication with another agent. Successful grounding of the novel concept, in this case, requires (i) perceptual knowledge of related categories and (ii) an ability to connect a description of the unseen category with that knowledge. In this talk, we propose a novel machine learning task based on this scenario. We present preliminary language generation and understanding model architectures and measure their joint performance on zero-shot visual classification.”