Groq Compliance Witnessing

v0.7.9 RELEASED – Groq adapter with latency witnessing. Available on PyPI and npm.

1. The Evidence Gap

Groq provides a developer-friendly API dashboard with usage metrics, rate limits, and model hosting details. These are useful for operations. They are not sufficient for compliance.

Here is the gap between what Groq gives you and what a regulated deployment requires:

Capability	Groq Provides	Compliance Requires
Usage tracking	Aggregate token counts, request volume	Per-inference cryptographic fingerprint
Model identification	Model name in API response	Immutable model hash tied to each decision
Latency metrics	Dashboard averages, rate limit headers	Per-call latency anchored to the specific inference
Audit trail	API key activity logs	Decision-level evidence: what went in, what came out, when
Tamper resistance	Provider-controlled logs	Independent witness anchor verifiable without provider access

Groq's dashboard tells you that 50,000 calls were made last month. SWT3 tells you exactly what each call contained, which model responded, and proves the record has not been altered. The dashboard is operational visibility. The anchor is audit evidence.

2. Witnessing Groq Inference

SWT3 supports two patterns for witnessing Groq inference: the native Groq SDK adapter and the OpenAI-compatible SDK with Groq's base URL. Both produce identical witness anchors.

Python (Groq SDK)

from groq import Groq
from swt3_ai import Witness
from swt3_ai.adapters.groq import wrap_groq

witness = Witness(
    endpoint="https://your-instance.example.com/api/v1/witness",
    api_key="axm_live_...",
    tenant_id="YOUR_TENANT",
)

client = wrap_groq(Groq(), witness)
# Or: client = witness.wrap(Groq())  # auto-detected

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Classify this support ticket priority."}],
)

witness.flush()

Python (OpenAI SDK)

from openai import OpenAI
from swt3_ai import Witness

witness = Witness(
    endpoint="https://your-instance.example.com/api/v1/witness",
    api_key="axm_live_...",
    tenant_id="YOUR_TENANT",
)

client = witness.wrap(OpenAI(
    base_url="https://api.groq.com/openai/v1",
    api_key="gsk_...",
))
# Auto-detected as Groq by base_url

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Classify this support ticket priority."}],
)

witness.flush()

TypeScript (Groq SDK)

import Groq from 'groq-sdk';
import { Witness, wrapGroq } from '@tenova/swt3-ai';

const witness = new Witness({
  endpoint: 'https://your-instance.example.com/api/v1/witness',
  apiKey: 'axm_live_...',
  tenantId: 'YOUR_TENANT',
});

const client = wrapGroq(new Groq(), witness);
// Or: const client = witness.wrap(new Groq());  // auto-detected

const response = await client.chat.completions.create({
  model: 'llama-3.3-70b-versatile',
  messages: [{ role: 'user', content: 'Classify this support ticket priority.' }],
});

await witness.flush();

TypeScript (OpenAI SDK)

import OpenAI from 'openai';
import { Witness } from '@tenova/swt3-ai';

const witness = new Witness({
  endpoint: 'https://your-instance.example.com/api/v1/witness',
  apiKey: 'axm_live_...',
  tenantId: 'YOUR_TENANT',
});

const client = witness.wrap(new OpenAI({
  baseURL: 'https://api.groq.com/openai/v1',
  apiKey: 'gsk_...',
}));
// Auto-detected as Groq by baseURL

const response = await client.chat.completions.create({
  model: 'llama-3.3-70b-versatile',
  messages: [{ role: 'user', content: 'Classify this support ticket priority.' }],
});

await witness.flush();

3. Latency as Evidence

Groq's speed claims are a selling point -- sub-100ms time-to-first-token, hundreds of tokens per second on the LPU architecture. For operations teams, this is a performance metric. For compliance teams, it is something more specific: a verifiable SLA artifact.

The adapter captures latency_ms for every inference call. This is not an average from a dashboard. It is the actual wall-clock measurement for that specific request, anchored to the same fingerprint as the inference content. Over thousands of calls, this creates a latency distribution that auditors can use to:

• Verify SLA compliance -- compare anchored latency values against contractual thresholds
• Detect degradation -- identify periods where latency spiked, correlate with incident timelines
• Validate capacity planning -- prove that inference latency remained within bounds during peak load
• Support vendor evaluation -- compare witnessed latency across providers using the same ledger

4. Streaming Witnessing

Groq's streaming API follows the OpenAI-compatible pattern. The SWT3 adapter fully supports streaming -- the witness anchor is minted after the stream completes, covering the entire response.

Python

from groq import Groq
from swt3_ai import Witness
from swt3_ai.adapters.groq import wrap_groq

witness = Witness(
    endpoint="https://your-instance.example.com/api/v1/witness",
    api_key="axm_live_...",
    tenant_id="YOUR_TENANT",
)

client = wrap_groq(Groq(), witness)

stream = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Summarize the quarterly report."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

witness.flush()

TypeScript

import Groq from 'groq-sdk';
import { Witness, wrapGroq } from '@tenova/swt3-ai';

const witness = new Witness({
  endpoint: 'https://your-instance.example.com/api/v1/witness',
  apiKey: 'axm_live_...',
  tenantId: 'YOUR_TENANT',
});

const client = wrapGroq(new Groq(), witness);

const stream = await client.chat.completions.create({
  model: 'llama-3.3-70b-versatile',
  messages: [{ role: 'user', content: 'Summarize the quarterly report.' }],
  stream: true,
  stream_options: { include_usage: true },
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);
}

await witness.flush();

4b. System Prompt Witnessing

Groq uses the same OpenAI-compatible message format. System prompts are messages with role: "system" in the messages array. The adapter extracts all system messages and hashes them independently as system_prompt_hash -- the same behavior as the OpenAI adapter.

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[
        {"role": "system", "content": "You are a ticket routing agent. Classify priority as P1-P4."},
        {"role": "user", "content": "Customer reports total service outage."},
    ],
)
# system_prompt_hash captures the routing instructions for this call

If your system prompt changes between deployments, the hash changes. This creates drift evidence without exposing the prompt content.

5. Model Hosting Transparency

Groq hosts open-weight models from leading providers. Unlike closed-model APIs where the model is a black box, Groq's model catalog consists of publicly available architectures running on proprietary LPU hardware. The adapter captures the exact model identifier from every response, creating a verifiable record of which model version served each inference.

Model	Model ID	Provider Tag
Llama 3.3 70B Versatile	llama-3.3-70b-versatile	Meta
Llama 3.1 8B Instant	llama-3.1-8b-instant	Meta
Llama Guard 3 8B	llama-guard-3-8b	Meta
Mixtral 8x7B	mixtral-8x7b-32768	Mistral AI
Gemma 2 9B	gemma2-9b-it	Google

6. Regulatory Evidence Mapping

Every witness anchor minted on Groq infrastructure maps to specific regulatory obligations. The evidence is not theoretical -- it is the artifact that addresses the requirement.

Obligation	Regulation	SWT3 Evidence	Procedure
Automatic logging of AI system operation	EU AI Act Art. 12	Per-inference fingerprint with model ID, latency, and token counts	AI-INF.1
Transparency for deployers	EU AI Act Art. 13	Response hash proving output content at time of generation	AI-INF.1
Deployer obligations for high-risk AI	EU AI Act Art. 26	Continuous monitoring evidence via anchored inference trail	AI-INF.1
AI risk management documentation	NIST AI RMF MAP 1.5	Provider identification (groq), model hash, dependency chain	AI-INF.1
Mechanisms to track deployed AI behavior	NIST AI RMF MN 2.2	Decision-level monitoring with latency and drift detection	AI-INF.1
Model risk management -- ongoing monitoring	SR 11-7 / SR 26-2	Continuous model behavior evidence via response hashing	AI-INF.1

7. Quick Start

Install

# Python (Groq SDK)
pip install swt3-ai groq

# Python (OpenAI SDK)
pip install swt3-ai openai

# TypeScript (Groq SDK)
npm install @tenova/swt3-ai groq-sdk

# TypeScript (OpenAI SDK)
npm install @tenova/swt3-ai openai

Minimal Example

from groq import Groq
from swt3_ai import Witness

witness = Witness(
    endpoint="https://your-instance.example.com/api/v1/witness",
    api_key="axm_live_...",
    tenant_id="YOUR_TENANT",
)

client = witness.wrap(Groq())  # auto-detected

response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "Hello"}],
)

witness.flush()
# One anchor minted: model, prompt hash, response hash, latency_ms

Two lines added to your existing Groq integration. Zero changes to application logic. Every inference call now produces a cryptographic witness anchor with latency evidence.

8. References

• Groq API Documentation
• Groq OpenAI Compatibility
• swt3-ai on PyPI
• @tenova/swt3-ai on npm
• Runtime Surface Governance Guide
• EU AI Act Compliance Crosswalk
• NIST AI RMF Crosswalk
• SWT3 SDK Documentation

Other Witnessing Guides

• Anthropic Claude Compliance Witnessing
• OpenAI GPT Compliance Witnessing
• Google Vertex AI and Gemini Compliance Witnessing
• LangChain and LangGraph Compliance Witnessing
• DeepSeek GPAI Compliance Witnessing