Reading Claude's Thoughts — The Pipeline

This lays out the pipeline for reading a model's internal thoughts: from the model processing input, to capturing activations, applying the interpretability lens, and reading the top concepts.

How it works

1. 1The model processes the input (e.g. 'what is the capital of France?').
2. 2Capture internal activations — a snapshot at a layer.
3. 3Apply the lens to compute how activations influence next-token probabilities.
4. 4Read the top concepts in the internal workspace right now.
5. 5The developer understands what the model is 'thinking' (e.g. France, Paris).

Key points

• The model predicts the next token at every step, which makes this readout possible.
• It reveals concepts before the model speaks or acts.
• Examples show it catching bugs, prompt injection or hidden goals.
• It's powerful but approximate — a research technique, not a perfect readout.

Why it matters

A concrete pipeline shows how interpretability can be operationalised — turning internal activity into readable concepts that help monitor and steer models.

Frequently asked questions

What makes reading thoughts possible?

The model computes next-token probabilities at every step, so internal activations can be mapped to concepts.

Is the readout exact?

No — it's an approximate, research-grade technique, but a powerful one.