OpenKarta
Conformance · signed badges

Pass the suite. Sign the badge. Get listed.

Every agent in the registry carries an HMAC-signed badge attesting to a recent conformance run. Auto-detected packs, closed enums, no manual review.

npm i @openkarta/conformance-tests ↗ View live badges ↗

How a run works.

1

Detect

discover() returns the agent's supportedItemTypes. The runner picks core + matching per-type packs automatically.

2

Execute

Every test exercises the wire format directly — closed-enum errors, TTL clamping, signature verification, lifecycle transitions.

3

Sign

A pass emits an HMAC-signed badge bound to the agentId, suite version, packs run, and timestamp. Tamper-evident.

4

Re-verify

Cron re-runs the suite daily against every listed agent. Stale badges auto-degrade; seven failures → delisted.

Test packs

core required

Required for every agent. Verifies discover, search, get, quote, checkout, status, cancel — including signature checks, error envelope shape, and TTL enforcement.

product auto-detected

Physical goods specifics: address validation, delivery modes, return-window correctness, return verb lifecycle.

stay auto-detected

check_in / check_out coherence, occupancy validation, multi-night quotes, no-overlap on inventory.

flight auto-detected

IATA code validation, fare class enums, passenger manifest required at checkout, refund / cancellation windows.

bus auto-detected

Operator + region pairing, seat-class enums, intercity route coherence, lifecycle through boarding.

service auto-detected

Slot-based availability, location mode (at-customer / at-provider / online / venue), reschedule via cancel + new quote.

Badge anatomy

Wire format
{
  "agentId":      "halcyon-shop",
  "suiteVersion": "0.3.0",
  "packs":        ["core","product"],
  "passed":       true,
  "testsPassed":  12,
  "testsFailed":  0,
  "ranAt":        "2026-04-26T14:00:00Z",
  "signature":    "hmac-sha256:..."
}
State machine
  • healthy — last sweep passed within 24h
  • stale — 3 consecutive failures, agent flagged for review
  • delisted — 7 consecutive failures, removed from public listing
  • unknown — newly verified, awaiting first sweep
Run the suite @openkarta/conformance-tests 
$ npx @openkarta/conformance-tests --base-url https://halcyon-shop.fly.dev

[discover] OK
[search ] 3 fixtures returned
[quote   ] signed token verified, ttl 180000ms
[checkout] order placed: ord_01J4...
[status  ] state=placed → in_transit
[cancel  ] receipt verified
[return  ] flow exercised

PASS  12/12  packs=[core, product]  suite=0.3.0
badge: hmac-sha256:a1b2c3...