Collaborative Construction and Communication with Minecraft

• Event: Seminar
• Lecturer: Julia Hockenmaier
• Date: 24 March 2021
• Duration: 2 hours
• Venue: Gothenburg

Virtual gaming platforms such as Minecraft allow us to study situated natural language generation and understanding tasks for agents that operate in complex 3D environments. In this talk, I will present work done by my group on defining a collaborative Blocks World construction task in Minecraft. In this task, one player (the Architect) needs to instruct another (the Builder) via a chat interface to construct a given target structure that only the Architect is shown. Although humans easily complete this task (often after lengthy back-and-forth dialogue), creating agents for each of this role poses a number of challenges for current NLP technologies. To understand these challenges, I will describe the dataset we have collected for this task, as well as the models that we have developed for both roles.