Catching AI Agent Protocol Regressions Before They Ship: A Diff-Based CI Gate

A REST API that silently drops an endpoint on deploy gets caught by a contract test, usually before it reaches production. An MCP server or A2A agent that silently drops a tool, loses its auth requirement, or downgrades its own protocol support has no equivalent gate — because there's no equivalent test suite. Every downstream agent integration finds out the hard way, at runtime, when a call it relied on stops existing.

Agent Protocol Inspector's scan diff exists to close exactly that gap: a structural comparison between two stored scans of the same target, producing a pass / warn / fail verdict a CI pipeline can actually act on.

What Gets Compared

diffScans takes a baseline scan and a current scan and compares all three protocols structurally — no new network calls, no re-probing, just a pure comparison of two already-fetched results:

Change	Verdict
A protocol's detection status regressed (confirmed → indicated or not_detected)	fail
An MCP tool, A2A skill, or ARD catalog entry was removed	fail
A declared auth mechanism disappeared entirely	fail
A tool/skill/entry was added	warn
An existing MCP tool's input schema changed	warn
An auth mechanism changed (without disappearing)	warn
Nothing meaningfully changed	pass

The asymmetry is deliberate. Removing something the agent could do before is a regression a downstream integration will break on — that's a fail. Adding new surface area is worth a human's attention but isn't inherently dangerous — that's a warn, not a build-breaker. A tool's input schema changing without being removed sits in the same bucket: worth reviewing, not worth blocking a deploy over on its own.

Wiring It Into CI

The verdict is only useful if a pipeline can actually act on it — and a bare curl call can't. An HTTP 200 response with {"verdict":"fail"} in the body still makes curl exit 0; nothing about that response naturally breaks a CI step. The fix is a couple of lines of jq:

RESPONSE=$(curl -sS -X POST https://contextiq.trango-compute.com/api/v1/agent-protocol-inspector/compare \
  -H "Authorization: Bearer $CONTEXTIQ_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"baselineScanId":"<last-deploy-scanId>","currentScanId":"<this-deploy-scanId>"}')

echo "$RESPONSE" | jq .

VERDICT=$(echo "$RESPONSE" | jq -r '.verdict')
if [ "$VERDICT" = "fail" ]; then
  echo "Agent Protocol Inspector: regression detected — failing build." >&2
  exit 1
fi

That's a real go/no-go gate: warn still prints the full diff for a human to read, but only fail stops the pipeline. This snippet is generated directly in the dashboard's "Compare against a prior scan" panel, next to the field where you'd paste in a baseline scanId to test manually before committing it to a workflow file.

A Practical Deploy-Gate Pattern

The workflow this is built for:

1. After every deploy, run a scan of your own MCP endpoint or A2A agent card and capture the returned scanId.
2. Store that scanId somewhere your next CI run can read it — a repo variable, a deploy artifact, a line in your release notes.
3. Before the next deploy goes live, run a fresh scan and diff it against the stored baseline using the snippet above.
4. On fail, the pipeline stops before the regression reaches users who were already depending on the tool or skill that disappeared.

This is the same shape as a contract test for a REST API — a stored "last known good" fingerprint, compared against every candidate release — applied to a surface (agent protocol conformance) that doesn't otherwise get tested at all.

What This Doesn't Catch

A diff only knows what changed structurally between two scans it was given — it has no opinion on whether the first scan was ever correct, and it can't tell you a regression happened if nobody ever stored a baseline to compare against. Pairing it with the conformance checks covered in our protocol scan deep-dive covers both halves: conformance catches a broken implementation on day one, and diff catches it breaking again on day two hundred.

Run a baseline scan with Agent Protocol Inspector today, and diff your next deploy against it before you find out from a support ticket instead.