AIs Playing Games with Language: Balancing Uncertainty, Truth, and Usefulness in Large Language Models

• Event: Seminar
• Presented by: Jordan Boyd-Graber Ying from the University of Maryland
• Date: 19 March 2025
• Time: 13:15-15:00
• Venue: Gothenburg University, Humanisten and online
• Address: Renströmsgatan 6, 412 55 Göteborg
• Room: J411
• Zoom link: https://gu-se.zoom.us/j/69780476534?pwd=Q9Uw2lu0zda8MsXkL08eGrqU64DMpp.1
• Slides: Jordan Boyd-Graber Ying 19.3.2025.pdf

Abstract

In this talk, I’ll talk about how we can use silly games to understand the strengths and weaknesses of AIs. We first begin with games that test memory: testing the recall of obscure facts. While AI has been viewed as superhuman at this task, it isn’t universally so. We show that a new measure of adversarial datasets (the gap between humans and computers) is decreasing but not yet closed, with computers still struggling on abstract reasoning and knowing when they know the correct answer. Given these disparate skill sets, we then analyze how we can best build human and computer teams to learn new facts and detect false statements. Finally, I close with a similar line of results for another silly language game, Diplomacy, where computers have still not reached dominance but can be used to assist human players think strategically and detect lies.

Bio:

Jordan Boyd-Graber is a full professor at the University of Maryland. He has worked on model evaluations for human-centered topic models, psychologically inspired leaderboards, human–computer machine translation, and question answering. He also contributed new models for improving generative models with RL, interactive approaches for question answering, topic models, and negotiations. He and his students have been recognized with paper awards at EMNLP (2023), IUI (2018), NAACL (2016), and NeurIPS (2009, 2015), and he won the 2015 Karen Spärk Jones Award and a 2017 NSF CAREER Award.

He previously was an assistant professor at the University of Colorado, Visiting Research Scientist at Google Zürich, and Praktikant at the Berlin-Brandenburg Akademie der Wissenschaften. His undergraduate degrees are in Computer Science and History at the California Institute of Technology, and he received his PhD from Princeton University. His Erdös number is 2 (via Maria Klawe), and his Bacon number is 3 (by embarrassing himself on Jeopardy!).

He lives in Silver Spring, Maryland with his wife, two roombas, three fish, two daughters, and their 外婆.