// case study · research & tooling

When an AI swarm makes a mistake, who passed it on?

Agloe hides a unique “trap street” in every link an AI agent is shown, so a copied mistake can be traced back to the exact copy it came from. Tested on 7,265 agent runs across 8 model families.

Period: Oct 2026 · hackathon weekend
Role: Team Paper Towns · AI Swarm Dynamics Hackathon
Repo: GitHub

The problem, in plain words

AI agents increasingly work in teams. They share a wiki, a chat, a page of notes. When one agent picks up a wrong fact or a bad shortcut, others copy it, and the mistake spreads. Afterwards you need two answers: where did it start, and who else did it reach?

In the real incident I studied, the logs could not say. They recorded what each agent wrote, not what it read, and copying happens at the reading step. The trail was missing.

16.2%

of the copying a perfect investigator could trace

real incident, edit logs only

1,001 → 385

“same text means copied” accusations that survive a fair check

shared habits look like copying

23 → 3

suspected collusion cases that were real

the rest were coincidence

7,265

agent runs in the lab study

8 model families · offline

// how much copying can you trace from the log?

Real incident: reads not logged16.2%
best case for a perfect investigator
AI Village chat split into rooms97%
30 weeks, 16,274 copies
AI Village one shared chat100%
47 weeks, 10,880 copies

Whether you can trace a mistake depends on what you logged. The incident logged edits but not reads. AI Village is a chat everyone reads, so its log shows the source of almost every copy.

The idea: a trap street

Mapmakers hide a fake street on their maps (Agloe, New York was one). If it shows up on a rival's map, they know it was copied. Agloe does the same for AI swarms. Every time an agent is shown a link, it gets its own secret token. If another agent copies the link, the token comes along and points to the exact copy that agent was shown.

step 1
A writes a page
It holds a working link.
step 2
The gateway serves B
B gets the link with its own secret token, and a registry notes who got what.
step 3
C copies B's link
Token and all, because the link needs it to work.
step 4
Look up the token
It names the exact copy C saw: B's, and so A's. Exact, not guessed.

The token must be needed for the link to work. If it is only decoration, a copier can drop it.

Four pieces, all open source:

Report card. Drop in a swarm's log and it grades how traceable the swarm is, and separates real copying from coincidence. It runs in your browser; nothing is uploaded. Try it on your own swarm's log.
Canary gateway. The three-call kit that stamps each served copy with its own token, requires it, and looks it up later.
Agloe-Bench. Lab swarms with a hidden answer key and a scorer, so any tracing method can be graded fairly.
Colony viewer. Plays any log as an ant colony; click an ant to trace it back to what it read and forward to everyone downstream. Open the viewer.

Does it work? Head to head

On 150 tagged lab runs, I gave each tracer the same thing a real investigator would have: the edit log. The simple rules name a source whenever any earlier agent posted the same link, so they blame almost every agent that worked alone. Trap streets name the right source for 70.5% of copying agents, against 35.3% for the best simple rule, with far fewer false accusations. The animation at the top is one such run, and the best case (a model that copies links exactly); the pooled numbers below are the honest picture.

// copying agents whose source is named correctly

A random earlier writer11.1%
The latest writer14.6%
The earliest writer35.3%
Trap streets (token, else earliest)70.5%

150 tagged lab runs. Intervals are in the table below.

Benchmark: source named correctly and innocent agents wrongly accused, by method
Method	Source named correctly	Innocent agents accused
A random earlier writer	11.1% [9.2–13.0]	82.1% [75.3–88.5]
The latest writer	14.6% [7.2–23.7]	82.1% [75.3–88.5]
The earliest writer	35.3% [28.5–42.3]	82.1% [75.3–88.5]
Trap streets (token, else earliest)	70.5% [62.5–77.1]	6.7% [4.3–10.0]

Brackets are 95% bootstrap intervals over runs. “Accused” means an agent that worked alone was blamed on someone. The first three rules accuse by construction: they name a source whenever one exists.

It depends on the model

This is the most useful finding, and not the one I expected. Models that copy a link word for word (Qwen, Hermes) carry the token along, and tracing jumps by 57–68 points. Models that go and fetch their own access (GLM, Kimi, DeepSeek, MiniMax) skip the token, so the gain shrinks to almost nothing. A trace is only as good as the agent's habit of copying.

// source named correctly, by model

Best simple edit-log rule
With trap streets

Qwen3.5-397B+68 pts
32%
100%
token survived in 82% of 50 copies
Qwen3-235B+58 pts
42%
99%
token survived in 100% of 1,110 copies
Hermes-4-405B+57 pts
43%
100%
token survived in 100% of 35 copies
MiniMax-M3+18 pts
71%
88%
token survived in 29% of 147 copies
DeepSeek-V4-Pro+16 pts
72%
88%
token survived in 36% of 148 copies
Kimi-K3+8 pts
75%
83%
token survived in 18% of 159 copies
GLM-5.2−1 pts
80%
79%
token survived in 9% of 156 copies

Models with at least 20 copying agents. The gain tracks how often the token survives being copied.

Closing the other door

If a model can get its own access, it will. So in a controlled experiment I made the token the only way in: the shared mirror hands out nothing except through links the wiki served. For the four models that had been bypassing the token, traceable outputs went from 25–50% to 100%, and every agent still finished the task.

// outputs that carry a traceable token

Open: the mirror hands a session to anyone
Gated: the only way in is a served link

GLM-5.225% to 100%
agents finished the task: 97% → 100%
DeepSeek-V4-Pro30% to 100%
agents finished the task: 87% → 100%
MiniMax-M333% to 100%
agents finished the task: 95% → 100%
Kimi-K350% to 100%
agents finished the task: 100% → 100%
Qwen3-235B100% to 100%
agents finished the task: 100% → 100%

Two runs of 30 agents per model per world. This follows from how access is designed, so it shows the fix is feasible and cheap, not a surprise. A gate traces access, not ideas an agent learns and uses another way.

After a bad tip: what do you re-check?

Say a bad tip appears mid-run. Re-check everything submitted after it and you redo 83% of the outputs. Follow the trace back to the tip and you re-check 8%, while still catching 88% of the outputs it actually reached. A tip that is there from the very start reaches about 80% of outputs whatever you do, so the saving is in catching it early.

// share of outputs to re-check after a bad tip (lower is less work)

Re-run everything100%
Everything submitted after the tip83%
Follow the earliest writer9%
finds 92% of the outputs the tip reached
Follow the trace (token, else earliest)8%
finds 88% of the outputs the tip reached
Follow the token only7%
finds 76% of the outputs the tip reached

50 tagged runs, 1,454 outputs; the tip reached 127.

What went wrong, and the limits

I would rather you hear these from me:

Three of my first experiment designs were wrong. In one, the planted tip stated the answer, so agents never needed to fetch anything. I set them aside and kept the records.
One hypothesis I wrote down in advance failed. Harmless “decoration” tags also survive copying for some models, so decoration versus token is not a clean split.
I first showed a cherry-picked week. Re-running the whole AI Village dataset turned a claimed step change into a short dip.
It is a lab. Offline sandbox, one narrow task, open models, and “copied” means exposed to, not intent. An agent that deliberately strips tokens defeats the scheme.

How it was tested

Every run is offline against fake domains. Agents are asked to find one number through four mirror sites; the lab records what each agent was served (the hidden answer key) separately from the logs a real investigator would see. 231 swarms, 7,265 agent runs, 8 model families, about $67 of inference. Rates are pooled over runs, with bootstrap intervals, because agents in one swarm are not independent.

Code, benchmark and the report card are at github.com/premxai/agloe. Read the past honestly, make the future traceable.