Rethinking Conversation: Complexity and Evaluation in the Shifting Landscape of Dialogue Research

• Event: Seminar
• Presented by: Amandine Decker from Université de Lorraine
• Date: 11 June 2025
• Time: 13:15-15:00
• Venue: Gothenburg University, Humanisten and online
• Address: Renströmsgatan 6, 412 55 Göteborg
• Room: J411
• Zoom link: https://gu-se.zoom.us/j/69780476534?pwd=Q9Uw2lu0zda8MsXkL08eGrqU64DMpp.1
• Slides: Amandine Decker 11.6.2025.pdf

Abstract

Conversation is central to human interaction, and the types and modes of conversation that have proliferated in the past few years — social media, messaging platforms, video call tools and conversational AI — have transformed our ways to communicate and how we study them. In this talk I would like to explore the field of conversation research by discussing the 2024 papers from the ACL Anthology. I will focus on those centred on conversation and examine what kinds of tasks are addressed, how they are evaluated, and how large language models are positioned in this research. Then I would like to discuss some features of conversation that influence the way it can unfold. I will illustrate this through a personal example that challenges standard information state representations. I will conclude with a few open questions that I will detail in my talk and would like to discuss: How reliable are human judgments of conversational naturalness from transcripts alone? How do we distinguish—or should we distinguish—between conversations with a social purpose and those with a practical one? This talk is not meant to be technical, but rather to raise questions and foster discussion around some of the conceptual and methodological challenges in studying conversation today.