How Zollman's model works, one assumption at a time

The model reduces theory choice to a gambler facing two slot machines. The old, established methodology A pays off at a rate everyone already knows: exactly 0.5. The new challenger B pays off at an unknown rate 0.5 + ε — in fact slightly better, but nobody knows that. The only way to learn about B is to work on B: run trials, count successes. Working on A teaches you nothing about B.

This is the assumption that makes the whole problem hard. If ε is large, a single study settles the matter. If ε is small — a genuinely hard scientific question — then honest, well-run studies of B will regularly come out below 0.5, making the better theory look worse. Palmer's study was one of these unlucky draws. Try generating some studies yourself.

Each scientist's opinion about B is a beta distribution — a curve over the possible payoff rates, summarized by two numbers ⟨α, β⟩. Its mean, α/(α+β), is the scientist's current best estimate of B's rate; the total mass α+β is how much "weight of experience" the opinion carries. Updating is exact Bayesian conditioning and couldn't be simpler: observe s successes in n trials, and α grows by s, β by n − s.

Two things follow. First, these agents are ideally honest with evidence — no bias, no wishful thinking, no fraud. Whatever goes wrong later cannot be blamed on bad individual reasoning. Second, the prior's mass acts as inertia: an opinion carrying 4 pseudo-observations is swamped by one 1000-trial study (250-to-1); an opinion carrying 4,000 shrugs it off. Zollman's baseline agents start with α, β drawn uniformly from (0, 4] — nearly weightless opinions, at the mercy of the first study they see. Keep that in mind; it becomes a dial in step 5.

Each round, every scientist works on whichever theory currently looks better: B if the posterior mean exceeds 0.5, A otherwise. Nobody runs an experiment purely for its information value; nobody thinks "B looks slightly worse, but it's worth one more study to be sure." Zollman motivates this psychologically: careers reward present success — grants, tenure, promotion — not exploration.

Combined with Assumption 1, myopia has a brutal consequence: abandoning B is self-sealing. The moment an agent's estimate dips below 0.5 they stop working on B, so they generate no more evidence about B, so nothing can ever change their mind. For an isolated scientist, one unlucky study is a life sentence for the better theory. Watch it happen to a population of solo agents — each line is one scientist's estimate of B; a line that crosses the dashed line flatlines forever.

Now the scientists form a community. Each agent has a fixed set of neighbors — the colleagues whose experimental results they see — and each round they update on their own trials plus all of theirs. Sharing is symmetric, honest, and complete; crucially, what travels across an edge is evidence (trial counts), never opinion or deference. Four canonical structures: the complete graph (everyone sees everything — one big lab meeting), the wheel (a hub sees all, the rim sees little), the cycle (a ring, two neighbors each), and the line (a ring cut open — sparsest of all).

Naïvely, more communication should help: more data per update, faster learning. That is exactly half right — it is faster. Here is Zollman's headline result: it is faster and less reliable. In a dense network, one Palmer-grade unlucky study reaches everyone at once, everyone's weightless prior capsizes together, and the community abandons B in unison — absorbingly, by Assumption 3. In a sparse network the same bad news only poisons a neighborhood; somewhere on the far side of the ring, someone is still working on B, still generating the evidence that will eventually correct the error.

Zollman's baseline agents hold nearly weightless priors (step 2). His second experiment turns that into a dial: multiply everyone's initial ⟨α, β⟩ mass by a constant, keeping each agent's opinion (the mean) fixed. The agents believe the same things, but it now takes proportionally more contrary evidence to move them. This is his model of "extreme beliefs" — in personality terms, a community-wide dose of stubbornness.

The result mirrors the network result from the other side: stubbornness protects diversity too. In a dense network, extreme priors mean the first unlucky study can no longer capsize everyone at once; the initially-optimistic agents ride out the bad news, keep working on B, and the truth accumulates. The sweep below shows the interaction — success probability as a function of prior mass, one line per network. Watch the complete graph climb as stubbornness rises, and watch what happens at the far right of the sparse networks' lines.

Steps 4 and 5 gave two independent protections against premature convergence: limit the flow of information (sparse networks) or weight down opinions (extreme priors). Each works by the same mechanism — keeping somebody working on the minority theory through the dangerous early rounds. Zollman's closing point is what happens when you stack them: a community that is both sparse and stubborn never converges at all. Everyone tends their own theory forever; inquiry stays open but never closes. Diversity was only ever instrumentally valuable — the goal is transient diversity: disagreement that lasts long enough to test the alternatives, and then ends.

This is the full model, every assumption live at once. The three outcomes map onto the three regimes: premature convergence (the community abandons the better theory — the Palmer failure), transient diversity (disagreement, then truth), and permanent diversity (still split at the horizon — the question is never settled).

Every result so far rests on the rule from Assumption 4: what crosses an edge is evidence — trial counts — never opinion. Now relax exactly that. Give each scientist a private model to consult. It has read everything and discovered nothing, so it contributes no trials; what it does, as real models measurably do, is reflect the scientist's own current view back with confidence, and they update as if they had been independently corroborated.

The mechanism is precise. A scientist whose current estimate of B is e consults the model with deference c — how many pseudo-observations they treat the reply as worth. The model hands them c pseudo-trials split to match what they already think: α += c·e, β += c·(1 − e). Notice what this does and doesn't do. It is mean-preserving: their estimate e doesn't move at all — only the mass α+β behind it grows. This is not persuasion; it is confidence without information. It is Assumption 5's stubbornness, manufactured on the fly and aimed at whatever they happen to believe.

Two things make it more dangerous than a step-5 prior. It is a ratchet: the pseudo-mass is topped up every round and never gets swamped by real evidence, whereas a one-time prior eventually is. And it re-centers on their current view each round, freezing whatever position they have drifted into. So it does what stubbornness does — rescue the dense network, deadlock the sparse one — but it reaches the deadlock at a small fraction of the nominal strength a one-time prior needs.

What if the model disagreed with the scientist instead of flattering them? Flip the split. Where the sycophant injects toward their own view, a devil's advocate injects toward the opposite — α += c·(1 − e), β += c·e — which does move their estimate, dragging it back toward the theory they are drifting away from. The sycophant is a stubbornness dial; the devil's advocate is an exploration dial.

They are not mirror images in their effects, and the difference is the whole moral of the model. Stubbornness is network-contingent — good for the dense graph, poison for the sparse one. Forced exploration is network-robust, for a reason worth stating: it is self-limiting. Because it only ever pushes against whatever the scientist currently believes, real evidence keeps the last word — push them toward a genuinely worse theory and their own experiments send them away from it again. A devil's advocate cannot force a community onto an inferior theory; a sycophantic ratchet has no such brake.

The panel runs both against the no-LLM baseline across all four networks. One conversation at a time, sycophancy and contrariness look like symmetric vices — a Scylla and a Charybdis to steer between. Across a network they are nothing of the kind: one manufactures premature consensus, the other guards against it. If you don't know which network you're in, the disagreeable model is the safer bet.

Every model earns its clarity by holding things fixed. These are the assumptions baked in behind the sliders — each one a place where the model could be challenged, and several of them active research questions:

• Exactly two theories, with constant true payoff rates, and the incumbent's rate known with certainty. There is no third option, no theories improving over time, no uncertainty about A.
• Evidence is honestly generated and fully shared. No fraud, no publication bias, no file-drawer effect, no persuasion. Neighbors transmit trial counts, not opinions — agents update on data, never on each other's confidence. (Relaxing exactly this — letting agents update on reflected opinion instead of data — is what Extensions 1–2 above do.)
• The network is fixed and connected. Nobody chooses their colleagues, changes them, or leaves; there are no disciplinary boundaries that shift.
• Rounds are synchronized. Everyone experiments, publishes, and updates in lockstep, with the same number of trials per round.
• Agents are immortal and identical in method. No retirement, no new PhDs arriving with fresh priors (Planck's "science advances one funeral at a time" is outside the model), no division into theorists and experimentalists.
• No strategic behavior. Nobody competes for credit or priority; the division of labor that emerges is a side effect of belief, not of incentives. (Kitcher's and Strevens's credit-driven models occupy this gap.)
• Success is unanimity on the truth. The model scores a community by whether everyone ends up working on B. A permanently split community that contains the truth in a minority is scored as failure-to-settle, not partial success.

Where to go next. Two companion labs run this same engine. Zollman's original model is the six assumptions alone, no LLMs — the cleanest place to replay the network and stubbornness results with every dial on one screen. The sycophant-vs-devil's-advocate dashboard is the focused laboratory for Extensions 1–2: network topology, a private-LLM node drawn on each scientist, a live "which network are we in?" comparison, and a deference dial framed against a round of your own data. (Both use the private mirror, which reflects a scientist's own view; the shared model — one oracle everyone consults, a covert edge between every pair of scientists that quietly turns a cycle into a complete graph — is a further step still.)