Operations 2026-04-10 8 min read
How daily conformance sweeps work.
A walkthrough of the cron worker, the test runner, the badge signer, and the lifecycle that takes a flapping agent from healthy to stale to delisted.
The cron
A Cloudflare Worker runs every six hours. It pulls the list of verified agents from D1, partitions them across queue producers, and emits one verification job per agent. Workers are stateless; D1 owns the truth.
The runner
Each job pulls @openkarta/conformance-tests, hydrates fixtures, and exercises the agent over the wire. The runner auto-detects supportedItemTypes from discover() and selects core + matching per-type packs.
The signer
On a passing run, the runner emits a JSON badge — {agentId, suiteVersion, packs, passed, ranAt} — and signs it with the registry HMAC key. The badge is stored in D1 and exposed via the registry API.
The lifecycle
A passing run resets the failure counter to zero. Three consecutive failures move the agent to stale (still listed, badge degraded). Seven consecutive failures move the agent to delisted (removed from the public listing). A successful run resets the counter and restores the public listing.
What the sweep cannot tell you
It cannot tell you whether a merchant is fulfilling well, whether their reviews are accurate, or whether their inventory matches what is on the wire. The badge attests only to protocol conformance — not commercial quality.