A Non-Anthropomorphized View of LLMs by Halvar Flake

Halvar Flake (Thomas Dullien), a reverse engineer and security researcher, uses his blog ADD / XOR / ROL to make a narrow, technical case against how AI safety discussions talk about language models. His starting complaint: serious people discuss "alignment" as though a piece of software might develop something like a will of its own.

The post builds its argument from the mechanics. Tokens map to vectors in a high-dimensional space, and a piece of text becomes a path through it, one word at a time, out to the model's context length. A trained model with a fixed seed is, in his framing, a mapping that takes one such path and returns the next point on it, nothing more mysterious than that. He extends the same lens to "alignment," recasting it as the unsolved, largely mathematical problem of bounding how often an undesirable sequence gets generated, rather than a question about a model's values or intentions.

He is upfront that the piece is opinionated rather than empirical: no data, no experiment, just an argument laid out paragraph by paragraph, including his guess at why researchers who believe they might be building something mind-like resist this framing. He closes by comparing LLMs' likely real-world impact to electrification rather than to anything resembling a new kind of being.