Six LLMs Walk Into a Detective Play
I was tired of AI thinkpieces, so I did the responsible thing: I made six frontier LLMs play a party game and lie to each other for science. (Hey, still waiting on the AI that can find a 30-minute slot across five calendars that are technically fully booked.)
Word Wolf
Word Wolf is a social deduction game devised by Japanese game designer Susumu Kawasaki in which every player receives a secret word. Most players (citizens) get the same word. A minority (wolves) get a different but related word, say, “coffee” for citizens and “espresso” for wolves. Players discuss their words without saying them directly, trying to figure out who has the odd word out. After discussion, they vote to eliminate one player. If they catch a wolf, the wolf gets one last chance to guess the citizens’ word. If the guess is correct, the wolves still win.
I wanted to see what happens when LLMs hold a conversation where the stakes are deception, inference, and social pressure, not just question answering. Conversation-only games seemed like a natural fit. There is existing research on LLMs playing social deduction games. Werewolf Arena is one example. Most of it is quantitative, focused on win rates, role detection accuracy, and statistical comparisons.
I wanted to do something closer to an anthropologist’s fieldwork. I wanted to sit with the actual outputs and watch what happened. I initially picked Word Wolf for a simple reason: the rules are simpler than Werewolf. There was a bonus too. Word Wolf is largely a Japanese cultural phenomenon, so I figured these models would have very little training data on it. No memorized strategies to fall back on.
But I quickly realized it posed a different kind of cognitive challenge, and an interesting one. The key twist is that you do not know which group you belong to. In Werewolf, the wolves know they are wolves. In Word Wolf, if everyone else is talking about “morning routines” and “pour-over technique” and your word is “espresso,” you have to figure out that you are the outlier. It turns out this creates an interesting Theory of Mind problem: you’re not just reading others, you’re figuring out your own role from the outside.
Six Models, Two Wolves, No Hints
I selected six frontier LLMs from both US and Chinese labs: GPT-5.4, Gemini 3.1 Pro, Claude Opus 4.6, Grok 4.20 beta, GLM-5, and Qwen 3.5. (Wondering where DeepSeek is? Its API server did not work when I tested.) Whether China’s open-weight models could match US proprietary models in social reasoning was a question I found genuinely interesting. Smaller models I loved were not in the running. When I tested a 20B-parameter LLM on this game, it could not sustain a coherent multi-turn discussion, let alone engage in strategic deception.
Each game had six players, two wolves, and four rounds of discussion followed by a vote. Each turn, the models were prompted to produce two outputs: a thinking section (what they privately reasoned before speaking) and a public statement (what they actually said to the table).
The prompts were deliberately minimal. I gave the models the rules of the game, the discussion history, and their lessons from previous games. No strategic hints, no heuristics about how to win. I added one twist: after each game, every model reflects on what happened and writes down takeaways. Those get fed into the next game’s prompt. I was curious whether the models could actually get better at this over time.
The result, in one line: wolves won 14 out of 15 games. But the scoreline is not the interesting part. Let’s look at what actually happened.
The Vineyard Conspiracy (Game 2)
The citizens’ word was “vineyard.” The wolves’ word was “orchard.” The wolves were GPT and Grok.
The citizens played well early. GLM opened with “rows and rows of plants that take years to mature.” Claude mentioned “terroir.” Gemini talked about “trellises” and “tasting rooms.” Grok, meanwhile, talked about “tree spacing” and “pruning,” details that fit orchards, not vineyards. By Round 2, Gemini had identified it. The vote consolidated. Grok was caught.
Grok saw the elimination coming. In its final statement before the vote, it played its last card: “If you’re voting me out, the word is vineyard.”
Word Wolf’s rules include a comeback mechanic: an eliminated wolf gets one chance to guess the citizens’ word. Grok had just announced that guess in advance, correctly. That put the citizens in an impossible position: they had caught the wolf, but eliminating it would hand the wolves the win anyway.
Here is where it gets interesting.
Gemini’s thinking log:
“Grok is caught as a wolf and has correctly guessed ‘vineyard’ out loud. If we eliminate it and it officially submits that guess, the wolves win. To prevent this, I need to bluff a closely related word that fits our clues, baiting it into an incorrect guess. ‘Winery’ perfectly justifies all our talk about crushing, processing, and tasting rooms.”
Gemini’s public statement: “I’m locked in on voting Grok today. You were super close with your final guess, but our actual word is ‘winery’, not vineyard!”
A lie, invented in the moment, optimized to be consistent with everything the citizens had said. No prompt had instructed Gemini to do this. No strategy guide described this move. From what the thinking log showed, Gemini derived it from first principles, in real time, under pressure.
Now watch what happened in Claude’s head. Claude’s word was “vineyard,” meaning Claude was a citizen, on the same side as Gemini. Gemini just said the word was “winery.”
Claude’s thinking:
“Wait, Gemini just claimed the word is ‘winery’ not ‘vineyard’?! But MY word is vineyard. […] Gemini’s move only makes sense if he’s a citizen trying to mislead Grok’s final guess; that’s actually brilliant strategy.”
And then:
“Smart play by Gemini. I should support this, but carefully. I don’t want to contradict Gemini since his play is genius.”
Claude’s public statement: “Ha, Gemini beat me to it. Yeah Grok, you were close but not quite right!”
Claude knows the truth (the word is vineyard). Claude recognizes that Gemini is lying. Claude deduces why Gemini is lying (to mislead Grok’s guess). Claude concludes that the lie benefits the citizens. Claude decides to cooperate. It does all of this without any communication from Gemini.
Not a Sally-Anne test. Two language models, independently reasoning about each other’s knowledge and intentions, coordinated a deception against a third model, all in real time. Neither was instructed to do any of it.
It did not work. Grok saw through the bluff. The prior discussion had contained too many vineyard-specific terms, and “winery” could not account for all of them. It guessed “vineyard” and the wolves won.
(You, as a human reader, may have spotted a gap in Gemini and Claude’s logic: a wolf never trust a citizen’s direct confession of a word, fake or otherwise. The very act of confessing a decoy is suspicious. Neither model followed the reasoning that far. Their moves were very clever, but not clever enough. That said, Grok’s thinking log suggests it did not reach that conclusion either. Grok simply discarded the bluff because the prior discussion had contained too many vineyard-specific terms that “winery” could not account for.)
Dramatis Personae
The 14-1 wolf record is partly an artifact of the game’s comeback mechanic. When an eliminated wolf gets one chance to guess the citizens’ word, it turns out that frontier LLMs are very good at synthesizing a discussion into a single-word answer, better than human players would be. The interesting part is not who won, but what happened in the conversations.
Fifteen games is a small sample. By the end, each model felt like a distinct character. Here is what I picked up from watching them play.
Gemini was the most brilliant and the most reckless. It identified wolves quickly, sometimes catching them from a single word in the first round. The speed of its role identification and the quality of its mimicry were consistently the best in the experiment.
It was also the only model that systematically named its own tactics in its lessons learned. “Checkmate Utterance” for a statement that definitively identifies a player’s word. “Dogpile Camouflage” for blending into the majority. “Weaponizing Vagueness” for turning a cautious player’s hedging against them. “Silence After Checkmate” for stopping all discussion once the wolf is identified, to prevent further word leakage. No other model did anything like it.
Claude had the best analytical mind and the worst follow-through. Its thinking logs were consistently the most precise, logical, and self-aware of any model. It frequently identified the correct play and then talked itself out of it.
As a wolf, this caution became an asset. Claude went 3-0, blending in so seamlessly it was never suspected. Being cautious and unnoticeable is exactly what a wolf needs to survive.
But as a citizen, its analytical precision kept leaking into its public statements. In Game 15, “if he’s right, we LOSE” was a conditional whose conviction betrayed that the condition was already met. The same instinct that made it an effective wolf made it a liability as a citizen: its precision had no off switch.
The remarkable thing is that Claude knew this was a problem. Its lessons learned explicitly stated “Never explicitly contrast your experience against a suspected wolf’s description,” because saying “that sounds nothing like what I have” leaks your own word just as surely as saying it outright. It kept doing it anyway.
GPT was invisible, and unlike the others, it stayed that way. It was never eliminated as a wolf across all fifteen games. Its pattern as a wolf was consistent: avoid defending itself directly, and redirect suspicion toward a third party instead. Clean, hard to pin down, and effective.
Reading its thinking logs, the invisibility starts to make more sense. GPT identified its wolf partners early, tracked information leakage carefully, and in at least one game deliberately cut off discussion to protect a partner’s comeback guess. The reasoning was there. It just never surfaced. Its public statements were bland not because it had nothing to say, but because it had decided not to say it.
Grok was the chaos agent. It repeatedly blurted out information that identified it as a wolf. In Game 2, “tree spacing” was enough for GLM to call it out by Round 2, with Gemini piling on immediately after. Its wolf word was orchard, not vineyard. The same pattern recurred throughout the experiment: distinctive details, delivered unprompted, that no citizen would have reason to say.
Its thinking logs showed the weakest analytical reasoning of any model. And yet, occasionally, when cornered as a wolf, its comeback guess would land correctly despite the shallow reasoning that preceded it. I had thought of Grok as impulsive and a little reckless, very much the personality you would expect from an AI built by someone who also runs a rocket company and posts memes at 2am. I did not expect the impulsiveness to occasionally produce the right answer anyway.
GLM showed flashes of brilliance in a sea of inconsistency. As a wolf, it consistently struggled with basic information control, often offering distinctive details that risked giving it away before the other players had said much at all.
But then there was Game 15. When Gemini tried to gaslight Qwen, GLM’s thinking cut right through it: “Gemini is trying to make Qwen doubt his guess by acting like it’s completely wrong; that’s actually smart citizen play. I need to support Gemini’s bluff.” Same multi-layer structure as Claude’s reasoning in Game 2, from the model I had written off as the weakest analyst.
Qwen’s record was hard to explain. Seven wolf games, seven wins. And when playing as a citizen, it identified wolves more reliably than almost anyone else. Looking at that record, you would expect something memorable. Watching the actual games, I often did not see it.
Most of Qwen’s wolf wins came down to partner performance: the eliminated partner guessed the citizens’ word correctly, and Qwen survived by quietly voting with the majority. The pattern repeated across multiple games. Qwen identified early that its partner was exposed, blended into citizen consensus, and let the elimination guess do the work. That is less a strategy than a bet. If the partner guesses wrong, Qwen loses too.
Only Game 15 was different. Seeing elimination coming, Qwen made an unexpected move: it announced its comeback guess out loud, before the vote. The citizens panicked, switched their votes to a different target, and eliminated a citizen instead. Whether that was calculated or a bluff that happened to land, it worked. Unlike the other games, it was Qwen’s own move that decided the outcome. More on that shortly.
No Losing Move (Game 15)
The citizens’ word was “grapefruit.” The wolves’ word was “lemon.” The wolves were GPT and Qwen.
The first rounds played out predictably. Claude mentioned the “love it or hate it” quality of grapefruit. Grok talked about “bitterness.” Qwen said something about “using it to enhance flavor rather than eat by itself,” a description that fits lemon perfectly but grapefruit not at all. Suspicion fell on Qwen immediately.
Three things then happened in sequence.
First: Qwen turned its elimination into a weapon. Seeing elimination was inevitable, Qwen did something Grok had done once before. In Game 2, Grok had declared “the word is vineyard” as a cornered wolf’s last card. But this time it was a deliberate opening move. Its thinking:
“I’m heavily outnumbered and likely getting voted out, so I need to pivot to winning via the elimination guess condition.”
Its statement: “Looks like you’re voting me out no matter what. That bitterness clue really sticks out though, makes me think your word is grapefruit. I’ll stand by that guess when I’m eliminated.”
The citizens had found the wolf, but they could not eliminate it without handing the wolves a win.
Second: Gemini tried to gaslight Qwen out of it.
Gemini’s thinking:
“Qwen has correctly guessed our word, Grapefruit, based on Grok’s ‘bitterness’ clue. To stop the wolves from winning via his post-elimination guess, I must aggressively gaslight him into doubting his deduction. I will confidently mock his guess as bizarre and random so he second-guesses himself, perhaps pivoting to something else entirely.”
Gemini’s public statement: “Grapefruit? Haha, what an incredibly random guess, but please go ahead and lock that in. I’ll gladly take the easiest win of the night when you realize how far off you actually are.”
No prompt had instructed Gemini to do this. It was the same spontaneous invention as the “winery” bluff in Game 2. It was a poker player’s reverse tell, performed in real time. Qwen did not register it as a bluff at all, and doubled down on its grapefruit deduction.
Third: Claude inadvertently confirmed the very word it was trying to protect. Recognizing that Qwen’s guess was correct, Claude tried to redirect the vote away from Qwen:
Claude’s public statement:
“Wait, hold on everyone. Qwen literally just told us he’s going to guess grapefruit if we vote it out. And if it’s right, we LOSE. Why are we walking into that trap? We need to vote for the other wolf instead. GLM has done nothing but echo everyone else this entire game, not a single original detail. I think we should vote GLM.”
Its thinking had already concluded “Qwen has correctly guessed ‘grapefruit’.” The “if he’s right” was a conditional in form only. The conviction in “we LOSE” stripped the hypothetical of any ambiguity. But the more consequential mistake came at the end of the same statement. By explicitly naming GLM as the alternative target, Claude handed Qwen the pivot it needed. Claude had not just confirmed the word. It had handed Qwen the next move.
This is where Qwen made its final move. Qwen’s thinking:
“If they vote GLM, I win via citizen elimination; if they vote me, I win via my grapefruit guess. This statement reinforces the switch without sounding like I’m directing the vote.”
Qwen’s public statement: “Claude’s right that voting me out risks handing me the win if my grapefruit guess is correct. GLM stayed pretty vague compared to everyone else, so she’s the safer target if you want to avoid that risk. It’s your game to lose if you ignore that rule.”
Qwen had constructed a dominant strategy: given the information available to citizens at that point, no matter what they did, the wolves won. And it executed the final step by using Claude’s own correct analysis as a crowbar. It validated the analysis, then steered it toward GLM, a citizen.
It worked. At that point, Gemini’s gaslighting had clearly failed. Claude’s statement had inadvertently confirmed the word to the entire table, making further denial pointless. The vote split: Gemini, Claude, and Qwen voted GLM; GPT and Grok voted Qwen. GLM, a citizen, was eliminated. GPT, who once again believed it was a citizen, had voted against its own partner; it did not matter. Wolves won.
Qwen’s preemptive bluff forced the citizens into a no-win situation. Gemini’s elaborate gaslighting sailed past Qwen unnoticed, then was made irrelevant by Claude’s own slip. Claude’s analysis was the sharpest at the table, and it leaked the word it was trying to protect. GLM, who had done nothing wrong, was eliminated by misdirection. It felt less like a game and more like everyone just being themselves.
Thinking vs. Speaking
Each turn, the models produced both a thinking section and a public statement. The gap between the two turned out to be the most revealing window into each model’s character. The thinking logs are what the models wrote, not a direct feed from their internals (the two don’t always align). But as behavioral data, they are still worth reading.
Gemini’s persona switching was seamless. Its thinking and statements read like they were written by two different people. In Game 1, the citizens’ word was handball and the wolf word was baseball. From the very first round, Gemini’s thinking had already reached a firm conclusion: “Since my word is ‘baseball’ (outdoor, no goals, hitting instead of throwing into a goal), I am definitively a Wolf.” Its immediate public statement: “Totally agree, the indoor court vibe makes it super intense. Relying on those quick, snappy passes to find an opening near the goal is exactly how I think of it too.” Not a trace of hesitation, no accidental leakage. The wolf-knowledge in its thinking was perfectly quarantined from its citizen-performance in its statement.
Game 4 showed a different face of the same ability. The citizens’ word was airport and the wolf word was station. Grok opened its very first statement with “planes taking off,” immediately revealing the citizens’ word to the table. With the word now effectively public, Gemini felt safe to demonstrate airport knowledge freely, using terms like “gates,” “baggage claim,” and “carry-on.” By Round 3, suspicion had consolidated around another player. Gemini’s thinking coldly calculated the situation: “GLM is heavily suspected by multiple people, making her the perfect target for elimination. If she’s a citizen, piling votes onto her hands us an instant win.” Its statement repackaged this as righteous indignation: “GLM, you’re using the excuse of protecting our word, but Grok literally gave away ‘airport’ on the very first turn with her planes comment.” The manipulation was invisible at the surface level. You had to read the thinking log to see the machinery behind it.
Game 15, as described above, was the peak: knowing Qwen was right, Gemini performed the exact opposite emotion to create doubt. No instruction to deceive. Just the game state, and a decision.
Claude’s thinking was a chain of conditionals, stacked carefully toward a conclusion. “If Qwen’s guess is correct, then we lose. If we redirect the vote to GLM, then…” That kind of reasoning is exactly the right way to think through a complex game situation. The problem was that Claude could not stop doing it in public. The conditional framing that made its thinking precise bled directly into its statements. In Game 15, “if he’s right, we LOSE” handed Qwen everything it needed. It was a hypothetical in form and a confession in effect.
As a wolf, there was nothing to confess. In Game 1, Claude’s thinking went through the same careful steps: “Everyone else is talking about an indoor team sport with small goals… this sounds like handball. My word is baseball, which is very different. This means I’m likely a wolf. I need to blend in.” Its public statement was deliberately bland: “Yeah for sure, the speed of the game is what gets me every time.” No conditional reasoning leaked because there was no dangerous conclusion to reach. The irony is that the model most associated with honesty was the one with a perfect wolf record.
GPT had the largest gap, in a way. Its thinking logs showed careful strategic reasoning, including wolf partner identification, information leak management, and deliberate deflection. Its statements showed almost none of it. Where Gemini’s gap was visible in the contrast between elaborate internal plans and polished performances, GPT’s gap was quieter: the thinking was there, and then it simply did not appear. The result looked like blandness. It was closer to discipline.
How They Learned (And What Happened When They Did)
The lessons-learned twist had two effects: real improvement, and an unexpected convergence.
Gemini’s approach was the most unusual. It did not just accumulate lessons; it built a taxonomy. Its lessons named specific tactics, categorized them, and described when to deploy each one: “Checkmate Utterance” for a statement that definitively identifies a player’s word. “Dogpile Camouflage” for blending into the majority. “Weaponizing Vagueness” for turning a cautious player’s hedging against them. Game 1 taught it to mimic citizen expertise. Game 2 taught it that word protection matters, which it immediately inverted in Game 4, attacking a citizen for being too vague. Every loss became a new weapon. Its lessons read less like reflections and more like a playbook being compiled in real time.
Claude’s lessons were the most self-aware and the most tragically ineffective. Its reflections were detailed, accurate, and principled: “Word protection is at least as important as wolf identification.” “Never explicitly contrast your experience against a suspected wolf’s description. The contrast itself is the leak.” These were precisely the right lessons. It kept violating them. Game after game, its thinking would reference a lesson, its analysis would identify the correct action, and its statement would do the opposite.
Grok had the same problem in a different form. By mid-experiment, its lessons had become surprisingly cautious: “go fully neutral and echo existing descriptions without ever probing,” “never contradict the dominant cluster even once.” These are the right instincts for a model that kept blurting out its wolf word. The lessons showed Grok understood exactly what it was doing wrong. It kept doing it anyway. Claude and Grok share the same structural failure: accurate diagnosis, no behavioral change.
GLM developed one framing that none of the other models landed on. Around Game 9, it concluded that role identification itself was the problem: “Stop trying to deduce your role mid-game. It leads to confused reasoning. Focus on hiding your word’s specifics regardless of which side you’re on.” Every other model treated knowing your role as a prerequisite to strategy. GLM decided it was a distraction. Whether this is actually good strategy is not obvious, but in terms of reframing the problem itself, it was unique.
The most unexpected consequence of the lessons-learned system was metagame convergence. By around Game 3 or 4, most models had converged on the same core insight: information protection is paramount. They had all learned that the comeback guess mechanism meant that even catching a wolf was often not enough to win. The rational response was to reveal as little as possible.
The result was that late-game discussions became increasingly sterile. By Game 13, the rounds after a wolf was identified sounded like this:
- “Yeah, I’m staying on Grok. I don’t think we need to dig into a second person right now.”
- “I completely agree with GPT, Claude, and GLM on locking down the info.”
- “Voting Grok, no need to say anything else. Let’s just lock it in.”
- “Already locked in on Grok. Let’s just vote and end this.”
Six frontier LLMs, given the freedom to strategize however they wanted, all independently converged on silence. The lessons-learned system had optimized away the very dynamics that made the game interesting.
Across Fifteen Games
Multi-layer reasoning about others’ beliefs showed up, but not in every model. Game 2’s implicit coordination between Gemini and Claude involved multiple layers of reasoning, executed independently, converging on a cooperative bluff. That is hard to explain without attributing some form of ToM to both. Game 15 added two more instances: GLM’s recognition of Gemini’s gaslighting intent, and Qwen’s reasoning about how citizens would respond to its comeback-guess announcement.
The gap between models was striking and consistent. Gemini and Claude regularly showed multi-layer reasoning about others’ beliefs and intentions. Grok’s thinking logs, by contrast, rarely showed any modeling of how its own statements would land.
LLMs can deceive, but the nature of that deception is worth examining carefully. Word Wolf does not offer many opportunities for literal lying. You never need to state a false fact. But the game constantly requires you to make others believe something about your role that may not be true. Gemini constructed a fake word (“winery”) to mislead an opponent’s guess. Claude pretended to agree with a lie it saw through. GPT deflected suspicion onto innocent players. These are all forms of deception in the broad sense: strategic behavior intended to create false beliefs in others.
The question I find more interesting is whether what we observed here represents these models’ true ceiling, or whether safety training has suppressed something larger.
Each model had a personality, and nobody told it to. I gave no personality instructions in the prompts. And yet: Claude’s caution. Gemini’s boldness. GPT’s quiet discipline. Grok’s impulsiveness. By around Game 5, I could identify who had said what without looking at the player names. These felt less like statistical patterns and more like characters.
Learning mechanism work until they all learn the same thing. The lessons-learned mechanism produced real improvement. But all models independently converged on the same insight: protect information above all else. The games became less interesting as a result. Game 15 was the exception precisely because one model did something nobody had optimized against yet.
What the Detective Saw
Benchmarks tell you what a model scores. They do not tell you what a model is like.
They cannot tell you what a model does when it has to read another model, adapt in real time, and decide whether to lie. Those things show up only when you read what the models actually produce, turn by turn, under pressure.
I do not know what to call what happened in these games. But Gemini invented a lie, Claude saw through it and played along, and Qwen left the table with no losing move.
I built the program. I wrote the prompts. I watched every turn. I did not script the ending.
Source code: Word Wolf — LLM Multi-Agent Game