Why 'Thinking' Needs Sorting Out Before We Argue About LLMs

Aether Mug, the personal blog of writer and programmer Marco Giancotti, is where this piece works through a question the AI world keeps arguing past each other on: what does it actually mean to say a language model "thinks"?

Responding to Halvar Flake's essay arguing that large language models are best described as mathematical functions rather than agents that "behave" or "decide," Giancotti steps back from that specific dispute and asks a broader one: whether the word "thinking" even has one stable meaning worth arguing about. He draws a parallel to a century-old fight inside biology, where scientists have long debated whether it is acceptable to describe evolution in purpose-driven language, even though nothing about natural selection intends anything. Darwin himself worried about this exact wording problem, and more recent primatologists have settled on using human-like terms carefully and testably rather than assuming they map cleanly onto animal minds.

Giancotti applies that same caution to language models: calling their output "prediction" in one sentence and "intelligence" in the next is inconsistent, and neither label is obviously the correct one. He does not resolve the argument, and says so directly. What makes the piece worth reading is not a verdict but the reframing — "does it think" becomes a question about which abstraction level is doing useful work, not a yes-or-no fact waiting to be discovered, and that distinction is the actual subject of the essay.