A paper posted to arXiv on September 17 (2609.18996, by Joey Xiao of New York University and Haonan Huang of Princeton) reports a result no language-agent system had previously produced: a general-purpose coding agent, given a game description, a raw observation/action interface and an empty policy file, builds a complete game controller in a single autonomous session, then beats every fair built-in StarCraft II AI and wins full games of Civilization (Freeciv) by total conquest.
The paper introduces Gauntlet, a develop-freeze-evaluate framework that ports games from small arcades to full commercial-scale titles. The agent interacts with the live game, experiments on its own and engineers a standalone controller; the researchers freeze the result and score it on held-out instances with zero model calls during play. The point of the "bare contract" is emphasized: no strategy hints, no algorithmic priors, no hand-built architecture — the controller is written by the model itself. The authors call this compiled agency: development experience compiled into a persistent executable agent whose architecture was built by the model.
The evidence comes in two layers. On an unpublished procedural roguelike, held-out success rates range from 0 to 86 percent, with a sharp generational threshold: every observed session of a newest-generation system outperforms the best session of its predecessor. At full-game scale, a compiled raw-API controller defeats every fair StarCraft II built-in AI plus two cheating variants, and single-session programs win complete games of Civilization on held-out seeds. The authors note this was achieved at modest rates against novice AI, but the genre itself is the milestone: no prior language-agent system had won full games of this kind standalone, without per-turn model calls and a hand-crafted tactical layer.
The limitations deserve equal space. The paper states plainly that win rates against novice AI are modest. It is a preprint, not peer-reviewed. The full commercial-scale game tested is Freeciv — the open-source variant of Civilization — not the commercial original. The numbers are self-reported, with no third-party replication. The standard skepticism about agent benchmarks applies: a single game genre, fixed map seeds, and a long way from anything resembling general intelligence. What the paper calls a "develop-freeze-evaluate" loop is also a narrow slice of agency: the agent gets one clean shot at one game, not a changing environment with shifting objectives.
What is notable is not that an AI won games. It is how it won: the controller needs no per-turn model calls and no hand-crafted tactical layer during play. That is the first demonstration that a coding agent's capability can be detached from the live reasoning loop and exist as a compiled program. For anyone deploying agents, this touches the cost structure directly — compile once, run many times, instead of paying for a model call at every step. Frozen programs are also more inspectable than a chain of tool calls; the paper makes a point of saying the frozen programs are readable and auditable, which matters for safety review in a way that opaque runtime reasoning does not.
Whether the compile path — development experience turned into executable software — scales beyond fixed environments is the real question the paper leaves open. For now this is a one-time victory inside a controlled game world. But the direction is worth watching: if frontier coding agents can really leave behind the controllers they write, the default shape of an agent may shift from conversational reasoning toward shippable software — and the economics, and the safety conversation, change with it.
[1][2]