LLMs as Information Processing Machines

POINTS
1

MODE
Workshop

TYPE
Knowing

KIND
SoCIETIE KNOT

THREAD
Enabling Technologies

Summary

In this hands-on workshop you will train and use your own language model from scratch - with just pen and paper and a bit of dice rolling. Through interactive exercises and guided discussions you’ll see how language models (even large language models like ChatGPT) are fundamentally information processing machines, turning language inputs into language outputs. The workshop builds from basic principles to more complex applications through an exploration of language modeling as a probabilistic process of predicting “what token comes next” in a sequence, and ends with a poetry slam (for real).

Preparation

No pre-reading or prior technical knowledge is required—just bring your curiosity about how information theory shapes our understanding of human and technological communication.

What participants will learn

By the end of this workshop participants will understand how modern language technologies like autocomplete, predictive text, and even AI systems fundamentally operate by learning statistical patterns in language to make predictions about upcoming words or tokens.

Completion

With your newly constructed language model, in the final part of this workshop you'll use it to participate in a poetry slam - dramatic readings of newly-generated language model poetry. Your participation in this activity is your completion activity.

---

Written by

Ben Swift

Created by

Ben Swift