You can't join this event anymore

Hosted by

Guests

Location

About

In this seminar, organized by Carlos Zednik and Visiting Professor Cameron Bucker, we will explore the structure and operations of state of the art Large Language Models (LLMs) like OpenAI’s ChatGPT and AnthropicAI’s Claude. The discussion will adopt a multidisciplinary perspective, integrating computer science, philosophy, and the life sciences like psychology and biology. We will discuss various methods to evaluate and explain the performance of current-generation LLMs, consider methods to reveal what LLMs do or do not represent about the world, and evaluate the suitability of current LLMs or near-term extensions of them as models of human intelligence, cognition, agency, and reasoning. Cameron Buckner, Associate Professor in the Department of Philosophy at the University of Houston, is a guest of Carlos Zednik, Assistant Professor of Philosophy & Ethics, Department IE&IS, TU/e. PROGRAM ------- Day 1 | We 26 June | 14:00-16:00 Reading A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates Guiding questions 1. What is the Blockhead thought experiment? Does it doom purely behavioral approaches to understanding deep-neural-network-based AI? 2. What is the problem of memorization in LLMs? What kinds of methodologies can we use to control for it? 3. Does the training objective of LLMs—autoregressive next-word prediction—mean that these systems cannot represent word meanings or build models of the world those words are about? 4. How might LLMs represent word meanings in vector spaces? What aspects of representation are easy or difficult to capture in vector space models? 5. What is the “redescription fallacy”? How can we decide whether a redescription of a network’s operations in low-level terms (e.g. linear algebra or matrices) disqualifies them from serving as good models of cognition or understanding? 6. What is the most important modeling target for neural-network-based AI…e.g. compositionality, grammatical output, language understanding and grounding, world models, and cultural knowledge transmission? Which of these have current-generation LLMs clearly satisfied? Which remain the most challenging for current systems? Day 2 | Th 27 June | 14:00-16:00 Reading A Philosophical Introduction to Language Models - Part II: The Way Forward Guiding questions 1. How are benchmarks used to assess the performance of LLMs? What are some problems with relying on benchmarks? Which are the most serious, and which can be overcome? 2. How might mechanistic approaches to explanation offer a new route to understanding the “black box” of LLMs? What requirements must an explanation meet to satisfy mechanistic or causal approaches to explanation in philosophy of science (e.g. Woodward)? 3. Can probing and attribution methods tell us what an LLM represents? Why or why not? 4. Do interventionist methods to interpretability address the concerns with probing and attribution methods? Are there other concerns we might have about these very new methods for understanding the operations of LLMs? Are these methods appropriately disconfirmable and responsible to empirical evidence? 5. What is the most promising way to tweak or extend the prospects for current-generation LLMs as models of human cognition, reasoning, or agency? 6. Might LLMs be “alien intelligences”? Is this a useful way to think about their operations? Why or why not? Cameron Buckner Cameron Buckner is an Associate Professor in the Department of Philosophy at the University of Houston. His research primarily concerns philosophical issues which arise in the study of non-human minds, especially animal cognition and artificial intelligence. He began his academic career in logic-based artificial intelligence. This research inspired an interest into the relationship between classical models of reasoning and the (usually very different) ways that humans and animals actually solve problems, which led him to the discipline of philosophy. He received a PhD in Philosophy at Indiana University in 2011 and an Alexander von Humboldt Postdoctoral Fellowship at Ruhr-University Bochum from 2011 to 2013. Recent representative publications include “Empiricism without Magic: Transformational Abstraction in Deep Convolutional Neural Networks” (2018, Synthese), and “Rational Inference: The Lowest Bounds” (2017, Philosophy and Phenomenological Research)—the latter of which won the American Philosophical Association's Article Prize for the period of 2016–2018. Cameron just published a book with Oxford University Press that uses empiricist philosophy of mind (from figures such as Aristotle, Ibn Sina, John Locke, David Hume, William James, and Sophie de Grouchy) to understand recent advances in deep-neural-network-based artificial intelligence.