Public defence of Vladislav Maraev's doctoral thesis

• Event: PhD Defence
• Presented by: Vladislav Maraev
• Date: 29 August 2022

Cordially welcome to the public defence of Vladislav Maraev’s doctoral thesis on Monday 29 August, at 15:15 in room J439, Humanisten. The proceedings can also be followed online, Zoom link: https://gu-se.zoom.us/j/66086973989#success The title is “Who is laughing now? Laughter-infused dialogue systems”.

Doctoral candidate: Vladislav Maraev, University of Gothenburg

Supervisor: Christine Howes, University of Gothenburg Assistant supervisor: (1) Staffan Larsson, University of Gothenburg, (2) Jonathan Ginzburg, Université de Paris

Opponent: Professor Matthew Stone, Rutgers School of Arts and Sciences

Committee: Professor Gabriel Skantze, Kungliga tekniska högskolan, Stockholm Doktor Magdalena Rychlowska, Queen’s University, Belfast Professor Alexandra Weilenmann, University of Gothenburg

Chair: Doktor Ellen Breitholtz, University of Gothenburg

Date: 2022-08-29

Title: Who is laughing now? Laughter-infused dialogue systems

Abstract: T his thesis paves the way for including laughter in spoken dialogue systems in a domain-general and linguistically valid way using computational linguistics tools and methods. The thesis is concerned with three main areas.

The first area is the placement of laughter in relation to speech and other behaviours. We show that convolutional and recurrent neural networks can effectively predict laughs from dialogue transcripts, whereas human perforance in this task is significantly worse. Such prediction models allow dialogue systems to predict user laughter and, if needed, put system laughter in an appropriate place. Furthermore, we look at laughter placement in relation to gaze and show that laughter, performing different pragmatic functions, is related to different gaze patterns. These findings provide important implications for embodied conversational agents and social robots in regard to multimodal behaviour realisation and coordination.

The second area is concerned with interaction between laughter and the communicative intent of a user and system, as well as with the context in which the given intent occurs. We lay the groundwork for the central component of a spoken dialogue system by implementing a dialogue manager in a theoretically informed way using a proof-theoretic model based on linear logic. Our dialogue manager is then extended to support laughter functioning as feedback or a signal accompanying system feedback, and an answer to polar questions. Additionally, we look at how laughter can modify or form a dialogue act, and how laughter can improve Transformer-based deep learning models in the task of dialogue act recognition.

The third area is humour. Humour is intuitively related to laughter and some laughs highlight social norm violations and ironic statements, which bring laughter and humour closer together, even though humour is not necessary for laughter. We look at how humour is related to reasoning about social conventions and other learned and accommodated implicit assumptions, and how humour can be modelled in relation to creativity, implying situational and conversational creativity for artificial agents.

Full text: https://gupea.ub.gu.se/bitstream/handle/2077/72030/Thesis%20final_8aug.pdf