CLASP
The Centre for Linguistic Theory and Studies in Probability

Learning and Maintaining a Lexicon for Situated Interaction

If, when asked to “point at the mug”, a physically unimpaired person seems unable to identify a potential referent that is standing in front of them, we might hesitate to ascribe knowledge of the meaning of the word “mug” to them, whatever else they may be able to tell us about mugs(e.g., “wooden mugs were produced probably from the oldest time, but most of them have not survived intact.”, or “mugs are similar to cups”). And yet computational models of word meaning are good at the latter (e.g., by simply linking to knowledge repositories like wikipedia, where the previous sentence about wooden mugs was taken from), and fail at theformer.

In this talk, I will present our recent work at learning a lexicon for referential interaction, where the referential aspects of word meaning are modelled through perceptual classifiers taking real images as input. I show that this representation complements other computational meaning representations such as those derived from distributional patterns, as well as decompositional or attribute-based representations. The lexicon is learned through (observation of) interaction, and is maintained and defended in interaction.