Clocktower Bench: check Astra’s Elo 1859 and ~82% expected win vs Opus 5.5—not “AIs already lie”

Dwarkesh’s Clocktower Bench has frontier models play the social-deduction game Blood on the Clocktower against each other. The protocol is strict: in each game one model occupies every good seat and another occupies the Demon’s team, then the same deal is replayed with sides swapped so neither side keeps a lucky hand. The public board covers 11 models released in 2026 and 386 rated games.

OpenAI’s GPT-6 Astra leads at combined Elo 1859, with detection 1940 and deception 1806. In the site’s expected-win matrix, Astra’s chance against Anthropic’s Opus 5.5 is about 82%. The write-up says most of OpenAI’s yearly lead is on detection: detective skill rose about four times faster than Anthropic’s, while deception stayed roughly level. Check those public figures and the method note—not a scare headline that “AI already lies.”

Hard facts (https://clocktowerbench.com/ and data/ratings.json; accessed 2026-11-06, Asia/Shanghai):

• Scale: 11 models from 2026; 386 rated games; 1500 is the average Elo of those eleven.
• Method: Bradley–Terry / Elo fit using good rating, evil rating, a good-side advantage, and deal effects; 90% intervals from bootstraps over side-swapped pairs; overlapping neighbors are not treated as a reliable order.
• Equal settings: each lab’s “high” reasoning effort; shared 8,000-token turn ceiling and word limits.
• Leader: GPT-6 Astra (2026-09) combined 1859 (90% interval about 1779–1983); 27/27 wins as good, 17/27 as evil.
• Matchup: expected-win matrix lists Astra vs Opus 5.5 at 82%, and Opus 5.5 vs Astra at 18%. Opus 5.5 combined Elo is 1549.
• Extra findings: OpenAI towns keep updating Demon beliefs across days; Anthropic towns more often treat early results as settled. In 132 mixed-team extras (half Opus 5.5, half GPT-5.6 Sol), the mixed team rated below its members’ average, especially as evil.
• Artifacts: GitHub dwarkeshsp/clocktower; Hugging Face dataset dwarkesh/clocktower-games; video YouTube 1NQwfImTseU.

This is a controlled match benchmark, not a real-world fraud or social-engineering test.

Evidence strength: a public method page, downloadable game data, and a checkable Elo table. With 386 rated games, side swaps, and shared effort/token caps, “Astra 1859 / ~82% vs Opus 5.5” is a verifiable claim rather than vibes.

Concrete checks: confirm combined and split Elo in ratings.json; confirm the Astra×Opus 5.5 cell in the head-to-head matrix; spot-check raw games for true side swaps. If bootstrap intervals heavily overlap while prose claims a firm ranking, shrink certainty.

Steelman: splitting “catch a lie” from “tell a lie” into detection and deception is more usable than a vague “EQ” score. Locating OpenAI’s lead mainly in detection is itself a falsifiable mechanism claim.

Strongest steelman: social deduction forces multi-turn belief updates, coordinated speech, and voting—exactly where long-context agents often fail. Giving every seat on a team to one model, then forcing a side swap, cuts “lucky deal” noise.

Steelman risk: expected win rates come from the Elo fit, not naive per-pair win percentages; some pairs remain thin, so 90% intervals matter. Rules, prompts, and the 8,000-token cap are experimental conditions; they do not transfer cleanly to open voice tables or mixed human games.

“Lying” here means in-rules Demon-team speech, not a product-safety incident. Do not read the leaderboard as proof that models passed a real-world fraud test.

For evaluators, the next months turn on whether Astra’s 1859 still leads as new models enter, whether the detection–deception gap keeps widening, and whether mixed evil teams keep underperforming. For labs, this is a long-horizon dialogue stress test that does not lean on standard exam sets.

For readers, acceptance should rest on public Elo, intervals, and raw games—not the drama in a video voiceover.

What to check on Clocktower Bench is GPT-6 Astra’s combined Elo 1859, the ~82% expected win rate versus Opus 5.5, and whether “OpenAI’s lead is mostly detection” survives the public data—not a claim that AI already lies in the real world. If the numbers and method hold, this is a reviewable social-reasoning benchmark; if not, it remains a story-shaped leaderboard.

【No.2】