When evaluators need proof, not dashboards. How protocol-level evidence makes third-party AI assessment verifiable, portable, and vendor-independent.
Who this is for: Third-party AI evaluators, Notified Body assessors, C3PAOs, compliance auditors, and evaluation framework designers. This guide explains how protocol-level evidence works, why it matters for independent assessment, and what evaluators can verify without vendor contact. No prior SWT3 knowledge required.
SWT3 witness anchors provide cryptographic evidence that specific operational controls were active at a specific point in time. They do not replace the evaluator's independent judgment, professional expertise, or regulatory authority. Evaluators must determine whether anchored evidence is sufficient, appropriate, and relevant to their specific assessment context. The protocol provides evidence. The evaluator provides the conclusion.
Emerging pacing frameworks propose embedding third-party evaluators inside AI development organizations with employee-level access. The premise is sound: independent verification of safety commitments requires more than self-reported compliance. But access alone does not produce evidence.
An evaluator who observes a safety process can testify that the process exists. That testimony is valuable, but it has structural limitations:
The current evidence landscape for AI evaluation relies on vendor dashboards, screenshots, exported logs, and self-reported metrics. None of these are independently verifiable by a party that was not present when the evidence was generated. None survive chain-of-custody scrutiny in a regulatory proceeding.
Protocol-level evidence closes this gap. It produces cryptographic artifacts at the moment of compliance activity, binds them to a specific point in time, and makes them verifiable by any party using public mathematics and an open specification.
There is a meaningful distinction between product-level evidence and protocol-level evidence. The difference is not quality. It is architecture.
Evidence generated by and dependent on a specific vendor's tooling.
Evidence generated against an open specification, verifiable without vendor dependency.
The analogy is TLS certificates versus a vendor's internal security report. A TLS certificate is independently verifiable by any browser, any auditor, any automated system. It does not require contacting the certificate authority to confirm validity. The mathematics are public. The format is standardized. The verification is universal.
A vendor's internal security report may be thorough, accurate, and well-intentioned. But verifying it requires trusting the vendor, accessing the vendor's systems, and interpreting the vendor's format. It is product-level evidence.
SWT3 witness anchors are protocol-level evidence. The fingerprint formula is public:
SHA256("WITNESS:{tenant_id}:{procedure_id}:{factor_a}:{factor_b}:{factor_c}:{timestamp_ms}").hex()[0:12]
This formula is implemented with 100% parity across 8 programming languages (Python, TypeScript, Rust, C#, Ruby, Swift, Go, Kotlin), verified by shared test vectors. An evaluator can verify any anchor using only the published specification and a SHA-256 implementation available in every programming language and operating system on Earth. No vendor contact. No network access. No proprietary tools.
Every SWT3 witness anchor encodes structured information that evaluators can read directly, without a vendor dashboard or proprietary decoder.
SWT3-{TIER}-{PROVIDER}-{UCT}-{PROCEDURE}-{VERDICT}-{EPOCH}-{SHA256_12}
| Field | Meaning | Evaluator Value |
|---|---|---|
TIER | Deployment tier (E=Enclave, S=SaaS, H=Hybrid) | Indicates deployment architecture and data residency model |
PROVIDER | Infrastructure provider (VULTR, AWS, AZURE, GCP) | Maps to inheritance boundaries and shared responsibility |
UCT | Universal Control Taxonomy code | Cross-framework control family (ACC, AUD, CFG, AI, etc.) |
PROCEDURE | Specific procedure ID (e.g., AI-MDL.5) | Identifies exact compliance requirement being evidenced |
VERDICT | PASS or FAIL | Deterministic outcome based on factor evaluation |
EPOCH | Unix timestamp (seconds) | Cryptographically bound to the moment of evidence collection |
SHA256_12 | First 12 characters of the fingerprint hash | Tamper detection: recompute from factors to verify integrity |
Behind every anchor, three structured factors encode the actual compliance measurement. These factors are what make anchors actionable for evaluators, not just binary pass/fail tokens.
| Factor | Role | Example (AI-INF.2 Inference Latency) |
|---|---|---|
factor_a | Standard, threshold, or baseline | 5000 (SLA threshold in milliseconds) |
factor_b | Observed or measured value | 3200 (actual latency in milliseconds) |
factor_c | Delta, context, or qualifier | -1800 (negative = within SLA, PASS) |
Evaluators can read factors directly from the evidence record to understand what was measured, what the threshold was, and how the system performed. This eliminates the "black box verdict" problem where an assessor sees a pass/fail result but cannot determine the underlying measurement.
AI providers handling sensitive model telemetry face a tension: evaluators need evidence, but retaining raw inference data creates privacy and intellectual property risks. SWT3 clearing levels resolve this by separating integrity verification from data retention.
| Level | Name | What Persists | What Is Purged |
|---|---|---|---|
| 0 | RETAIN | Full forensic record | Nothing |
| 1 | FACTOR-ONLY | Anchor + factors | Raw evidence payload |
| 2 | ANCHOR-ONLY | Anchor only | Factors + raw evidence |
| 3 | SOVEREIGN | Anchor transmitted, then destroyed locally | Everything after transmission |
The critical property: the anchor fingerprint remains cryptographically verifiable at every clearing level. A provider operating at Level 3 (SOVEREIGN) has destroyed all raw telemetry, but the evaluator can still verify that the anchor was minted from real evidence at a specific point in time. Integrity does not require data retention.
Protocol-level evidence is only useful if the procedures being evidenced are relevant to the evaluator's framework. SWT3's Universal Control Taxonomy (UCT) maps 280 procedures to 77 regulatory frameworks through a single, machine-readable registry.
A single anchor can satisfy evidence requirements across multiple frameworks simultaneously. An evaluator assessing against the EU AI Act and a separate evaluator assessing against NIST AI RMF can both reference the same anchor, because the UCT registry maps the underlying procedure to both frameworks.
| Namespace | Focus | Procedures | Example |
|---|---|---|---|
AI-INF | Inference infrastructure | Latency, throughput, availability | AI-INF.2: Response latency against SLA threshold |
AI-MDL | Model integrity | Weights, adapters, quantization | AI-MDL.5: Model weight file hash verification |
AI-GRD | Guardrails | Input/output filtering, content policy | AI-GRD.1: Guardrail activation rate and bypass count |
AI-FAIR | Bias and fairness | Demographic parity, disparate impact | AI-FAIR.1: Bias metric against threshold |
AI-HITL | Human oversight | Review queues, escalation, intervention | AI-HITL.3: Reviewer identity verification |
AI-RAG | Retrieval-augmented generation | Source provenance, relevance scoring | AI-RAG.1: Retrieval source provenance chain |
AI-TOOL | Tool use | Function calls, API invocations | AI-TOOL.1: Tool call success/failure and latency |
AI-MCP | Protocol security | Transport integrity, tool discovery, consent | AI-MCP.1: MCP transport integrity verification |
The full registry is published, machine-readable (JSON), and versioned. Evaluators can query it programmatically to determine which procedures map to their assessment framework before beginning an evaluation.
The following table shows what evaluators can verify directly from anchor evidence, without vendor dashboards or proprietary tools.
| Procedure | What It Proves | factor_a | factor_b | factor_c |
|---|---|---|---|---|
| AI-MDL.5 | Model weights match expected hash | Expected SHA-256 prefix | Observed SHA-256 prefix | Match flag (1=match, 0=mismatch) |
| AI-GRD.1 | Guardrails were active during inference | Policy version identifier | Activation count | Bypass count (0 = no bypasses) |
| AI-FAIR.1 | Bias metric was measured and within threshold | Threshold value | Measured value | Delta (negative = within threshold) |
| AI-HITL.3 | Human reviewer identity was verified | Reviewer credential hash | Review timestamp | Authorization scope |
| AI-DRIFT.2 | Model drift was measured against baseline | Drift threshold | Observed drift score | Consequence classification |
| AI-REV.1 | A previous anchor was formally revoked | Revoked fingerprint | Reason code (0-6) | Revoking authority hash |
| AI-TOOL.1 | Tool invocation was recorded with outcome | 1 (tool called) | Latency in milliseconds | 1=success, 0=failure |
| NHI-ROTATE | Non-human credential was rotated on schedule | Credential hash | Rotation timestamp | Days since last rotation |
The AI governance landscape is converging toward one of two structural outcomes. Both create demand for the same evidence infrastructure.
If capability-pacing proposals become regulation, compliance evidence becomes mandatory. Every organization developing frontier AI systems will need to demonstrate that safety evaluations occurred, that capability thresholds were respected, and that third-party evaluators confirmed adherence.
In this outcome, evaluators need standardized evidence that:
Protocol-level evidence is the only format designed specifically to satisfy all four requirements. Product-level evidence ties the evaluator to the vendor's tooling. Protocol-level evidence ties the evaluator to public mathematics.
If open-weight models continue closing the capability gap with proprietary systems, the evaluation challenge inverts. There is no central organization to embed evaluators inside. Open-weight models are downloaded, fine-tuned, and deployed by thousands of independent operators. The "employee-level access" model has no building to access.
In this outcome, compliance evidence must:
Protocol-level evidence is the only format designed to work natively in a decentralized deployment model. Anchors minted at the model release stage travel with the model card. Any party in the distribution chain can verify them independently.
Whether AI development centralizes under pacing frameworks or decentralizes through open-weight distribution, evaluators need evidence infrastructure that is independent of the entity being evaluated. The evidence format cannot be owned by the organizations being assessed.
Embedded evaluation frameworks assume a physical or organizational boundary: evaluators are embedded somewhere. Open-weight models -- AI systems whose trained parameters are published for anyone to download, fine-tune, and deploy independently -- dissolve that boundary. A model released under an open license may be running in 10,000 organizations within weeks of release. No evaluation framework can embed assessors in all of them.
Protocol-level evidence addresses this structural gap through three mechanisms:
The model provider mints witness anchors during training, evaluation, and safety testing. These anchors document that specific safety procedures were executed: bias measurement occurred (AI-FAIR.1), guardrails were tested (AI-GRD.1), model weights were integrity-checked (AI-MDL.5), human reviewers were involved (AI-HITL.3). The anchors are published alongside the model card and weights.
Any organization deploying the model can verify the provider's anchors using the published specification and SHA-256. No vendor contact required. No API key. No subscription. The mathematics are universal. If the anchor fingerprint matches the recomputed hash, the evidence is intact. If it does not, the anchor has been tampered with or the claimed factors are incorrect.
The deploying organization mints its own anchors for deployment-specific controls: inference latency (AI-INF.2), tool call integrity (AI-TOOL.1), credential rotation (NHI-ROTATE), drift measurement (AI-DRIFT.2). These deployment anchors are independent of the provider's training-time anchors but use the same protocol, the same format, and the same verification process.
The result is a two-layer evidence chain: the provider proves the model was evaluated before release, and the deployer proves the deployment is operating within defined parameters. Both layers are independently verifiable by any third party.
Open-weight providers face a specific tension: releasing model weights for public use while retaining proprietary training methodology. Clearing Level 3 (SOVEREIGN) resolves this by allowing the provider to prove that safety evaluations occurred without retaining or exposing the raw evaluation data. The anchor is transmitted to the evaluator or published in the model card, then all local evidence is destroyed. The fingerprint remains verifiable. The training methodology remains confidential.
Given an anchor and its factor record (provided by the assessed organization or retrieved from the compliance ledger), any evaluator can verify integrity:
tenant_id, procedure_id, factor_a, factor_b, factor_c, timestamp_msSHA256("WITNESS:{tenant_id}:{procedure_id}:{factor_a}:{factor_b}:{factor_c}:{timestamp_ms}")SHA256_12 fieldThis verification works in any programming language, any operating system, and offline. No network connection, no vendor API, no specialized software.
Factors are procedure-specific. The UCT registry documents what factor_a, factor_b, and factor_c mean for each of the 280 procedures. For example:
Evaluators do not need to interpret raw model telemetry. The factor matrix translates operational measurements into structured, comparable values.
A single procedure maps to multiple regulatory frameworks through the UCT registry. For example, an anchor for AI-FAIR.1 (bias measurement) simultaneously provides evidence for:
This eliminates duplicate evidence collection. An evaluator assessing against multiple frameworks can reference a single evidence chain.
SWT3 anchors embed natively into NIST OSCAL Assessment Results as observation records. Factors persist as OSCAL properties. This enables direct integration with eMASS, FedRAMP authorization packages, and automated compliance pipelines without format translation.
Evaluation infrastructure must be independent of the entity being evaluated. If the evaluated organization provides the tools that generate, store, and verify compliance evidence, the evaluation is structurally dependent on the evaluated. Independence requires that the evidence format, the verification method, and the assessment criteria exist outside the control of any single organization.
This is not a theoretical concern. It is the central structural challenge of any embedded evaluation framework. An evaluator with employee-level access to an AI lab can observe processes, review documentation, and interview staff. But if the evidence artifacts they collect are generated by the lab's proprietary systems, stored in the lab's infrastructure, and verifiable only through the lab's APIs, then the evaluation chain has a dependency on the evaluated entity at its foundation.
Protocol-level evidence breaks this dependency by placing three elements outside any single organization's control:
When evaluators need proof that safety commitments are being met, the proof must be as independent as the evaluators themselves. Protocol-level evidence is how that independence becomes verifiable.
No call to action required. Read the specification. Verify an anchor. Review the procedure registry. Form your own conclusion.