詩

Little Poet

a tiny GPT you can watch think

Click any character to see what the model was thinking when it wrote it.

Inside position –

Next-character probabilities

Attention · which earlier characters each head looked at

Residual stream · vector size after each layer

Embedding neighbours · characters with the most similar vectors

How it works

Everything below uses this model's real numbers.

1 Characters become numbers

2 Numbers become vectors

3 Attention: looking back

4 MLP and the residual stream

5 Softmax and temperature

6 How it learned