The lab piece A GPT in slow motion shows a language model at work — not as an animation, but for real: what you see there are the actual computation steps of an actually trained model. The brief behind it was one sentence: vivid, but not dumbed down. So it had to be a real model — small enough to run in your browser, transparent enough to show every step. An AI built it. This report tells how — and where the self-reference needed limits.
The same architecture, just tiny
The AI first wrote a training program — deliberately without ready-made AI libraries, every formula spelled out, so nothing stays vague. The model shares the architecture of its big relatives (GPT-2 style), just with 159,488 parameters instead of many billions. It trained on what this site already holds: 116,493 characters from 26 of its own texts, German and English.
Whether such a hand-written program computes correctly is not a matter of trust but of testing. So the trainer carries a mathematical counter-check: every derivative the program derives is spot-checked against a blunt numerical approximation — if the two disagree, the formula is wrong. Training only started once that check came back green. And so the model computes exactly the same in your browser as it did in training, a self-test travels with the exported weights: a pinned expectation, re-computed on load. If it doesn't match, the lab shows nothing rather than something wrong.
Where the human stepped in
The best correction came from watching: I noticed the model continuing German sentences with English words. The question that followed — would a “proper” GPT do that too? — led to a real improvement: the training data was grouped by language, and training windows stopped crossing text boundaries. The model learned more cleanly afterwards, and the remaining drift is explained right in the lab. That's what the human share looks like here: not typing, but watching, comparing, asking the right question — and deciding, in the end, what goes live.
The stumbles belong in the report too. A wiring bug left the lab dead after certain page transitions — the AI had marked an element as “done” before it was fully built; it was found only through insistent follow-up, after the AI's first explanation (“known rendering flakiness”) turned out to be plain wrong. And that the lab text contains only numbers taken from the actual training run is a rule, not a given — the temptation to write down a plausible number is real for an AI as well.
Self-referential, but checkable
An AI builds a mini AI so that people can understand AI — it sounds like a snake biting its own tail. The difference lies in the test protocol: the mathematical counter-check in the trainer, the self-test on load, numbers only from the training run, and a human signing off. That turns self-reference from self-congratulation into a tool with an open casing. How the model computes in detail is shown in the lab under “Under the hood” — layer by layer, in slow motion.
/compact — the essentials when context is scarce:
For the GPT lab, an AI trained a real mini language model (GPT-2 style, 159,488 parameters) on 116,493 characters from 26 of this site's texts — with a hand-written training program and no AI libraries. Trust doesn't replace testing: a numerical counter-check validated every formula, an exported self-test guards the browser execution, and the lab quotes only numbers from the actual training run. The human supplied the decisive corrections (spotting mixed languages in training, questioning a wiring bug) and the sign-off — self-reference only works with a test protocol.