How to Witness NVIDIA NeMo Guardrails with Independent Cryptographic Evidence

1. The Guardrail Evidence Problem

NVIDIA NeMo Guardrails is one of the most widely deployed runtime safety frameworks for LLM applications. It intercepts prompts, evaluates them against configurable rails (content safety, topic control, jailbreak detection), and blocks or modifies unsafe inputs before they reach the model.

The problem is not whether guardrails work. The problem is proving they were active.

When a guardrail blocks a prompt injection, the guardrail's own logs say "blocked 1 attack." That is self-attestation. Under EU AI Act Article 12, regulators require independent, non-repudiable records that safety measures operated as designed. Under NIST AI RMF Map 1.5, risk management systems must produce verifiable evidence of continuous operation. "Trust us, we blocked it" is not sufficient evidence for either framework.

The solution separates two concerns: the guardrail enforces the policy, and an independent witness records that enforcement happened. The guardrail is the bodyguard. The witness is the notary recording that the bodyguard was on duty.

2. Two Runtime Layers

AI runtime governance operates at two distinct layers. Confusing them creates both security gaps and compliance gaps.

The two layers are complementary. NeMo Guardrails adds safety. SWT3 adds evidence. Deploying guardrails without independent evidence is like deploying a firewall without audit logs. Deploying evidence without guardrails is like logging that no bodyguard was present.

3. Integration Pattern

NeMo Guardrails is Python-native. The integration wraps the guardrail's generate() call and witnesses both the guardrail evaluation and the model response.

Basic Pattern

from nemoguardrails import LLMRails, RailsConfig
from swt3_ai import Witness
import hashlib, json

# Initialize NeMo Guardrails
config = RailsConfig.from_path("./config")
rails = LLMRails(config)

# Initialize SWT3 witness
witness = Witness(agent_id="nemo-guardrailed-agent")

# Generate with guardrails, then witness the result
response = rails.generate(
    messages=[{"role": "user", "content": user_input}]
)

# Witness the guardrail evaluation
# Attribute name may vary by NeMo version (e.g. raw_config, config, etc.)
config_hash = hashlib.sha256(
    json.dumps(config.raw_config, sort_keys=True).encode()
).hexdigest()[:12]

witness.witness_guardrail(
    guardrail_id="nemo-guardrails",
    verdict="pass",              # or "block" if rails blocked the input
    config_hash=config_hash
)

# Witness the inference
witnessed_response = witness.wrap(response)

With Guardrail Verdict Detection

Response structure varies by NeMo Guardrails version. Consult your version's documentation for the exact log format.

# NeMo Guardrails returns a special response when rails block input
response = rails.generate(
    messages=[{"role": "user", "content": user_input}],
    options={"rails": ["input", "output"], "log": {"activated_rails": True}}
)

# Determine verdict from response
blocked = response.get("log", {}).get("activated_rails", {}).get("type") == "block"
verdict = "block" if blocked else "pass"

witness.witness_guardrail(
    guardrail_id="nemo-guardrails",
    verdict=verdict,
    config_hash=config_hash
)

if not blocked:
    witnessed_response = witness.wrap(response)

TypeScript (via NeMo REST API)

import { Witness } from '@tenova/swt3-ai';

const witness = new Witness({ agentId: 'nemo-guardrailed-agent' });

// Call NeMo Guardrails REST endpoint
const railsResponse = await fetch('http://localhost:8000/v1/chat/completions', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ messages: [{ role: 'user', content: prompt }] })
});
const result = await railsResponse.json();

// Witness the guardrail evaluation
witness.witnessGuardrail({
  guardrailId: 'nemo-guardrails',
  verdict: result.guardrail_action === 'block' ? 'block' : 'pass',
  configHash: configHash
});

// Witness the inference
const witnessed = await witness.wrap(result);

4. Procedure Coverage

5. Inline Security Partners

NVIDIA's NeMo ecosystem includes multiple inline security partners. The SWT3 witness pattern works identically regardless of which partner's classifiers sit in the guardrail pipeline.

The witness does not inspect the guardrail's internal logic. It records the verdict, the version, and the configuration. This means switching from native NeMo rails to a partner solution like Votal AI requires changing the guardrail_id parameter, nothing else. The evidence format is the same. The assessor reads the same anchors.

6. Quick Reference for Examiners

7. References

• NVIDIA NeMo Guardrails Documentation
• NVIDIA Dynamo Integration Guide
• Kimi K3 Compliance Witnessing (open-weights models needing deployer-side guardrails)
• DigitalOcean Inference Witnessing (serverless inference with model routing)
• CoSAI Risk Map Crosswalk (AI-GRD maps to CoSAI Input/Output Validation controls)
• AI Threat Detection Witnessing (who watches the security AI)
• SDK Documentation (9 languages, 21 adapters)
• Machine-Readable Crosswalks (JSON)