Oleus Audit
Oleus Audit records every prompt, tool call and approval your AI agents make in a hash-chained, append-only log, and maps entries to SOC 2 controls CC6.1, CC7.2 and CC8.1. It produces evidence for your audit. It does not make you SOC 2 compliant or certified.
Install the SDK
pip install oleus
Wrap your LLM client
from openai import OpenAI from oleus.llm import wrap_openai, OleusHeld client = wrap_openai(OpenAI(), "gr_live_xxxx", agent_id="support-agent") # wrap_anthropic(Anthropic(), token, agent_id=...) works the same way try: client.chat.completions.create(model="gpt-4o", messages=[...]) except OleusHeld as e: # a sensitive tool call is waiting for approval: do not run the tool print(e.approval_request_id)
Raw prompts and completions never leave your process. Only SHA-256 hashes are sent. PII flags are computed in the SDK and stored as pii_detected_client (client-asserted, not verified by the server).
Gateway (no wrapper)
Point your provider SDK at the Oleus gateway instead of wrapping it. Your provider key is forwarded to OpenAI/Anthropic and never stored; Oleus authenticates with x-oleus-token.
from openai import OpenAI
from oleus.llm import gateway
client = OpenAI(api_key="sk-...", **gateway("gr_live_xxxx", "openai", "support-agent"))
Or set OPENAI_BASE_URL=https://api.oleus.io/gateway/openai/v1 / ANTHROPIC_BASE_URL=https://api.oleus.io/gateway/anthropic and send x-oleus-token and x-oleus-agent headers. Held tool calls return HTTP 403 oleus_approval_required; if the audit write fails the gateway returns 502 oleus_audit_unavailable (fail closed). Streaming responses are buffered before being returned. The gateway sees plaintext in transit to compute hashes and flags, then discards it. Coverage of "every call" requires blocking direct egress to the providers.
Approvals
These tools require human approval by default: send_email, send_slack, refund, delete, update_billing, execute_sql, create_user, transfer_money. A call to one returns HTTP 202 with status: "pending_approval" and is logged as agent_tool_call_blocked until an admin approves it in the dashboard.
- Approvals are bound to one agent and one tool, single-use, and expire after 15 minutes.
- The approver is set by the server from the authenticated admin. Any
approved_byorsoc2_controlsent by a client is ignored. - Pass the approved id on the retry:
create(..., oleus_approval_id=id), or callauthorize_tool()before running a tool yourself.
Verify the chain
Each entry stores a per-org sequence number, the hash of the previous entry and its own hash, computed inside Postgres under a per-org lock. The application database role has no INSERT, UPDATE or DELETE on the log; rows enter only through append_agent_audit().
curl -H "Authorization: Bearer $TOKEN" https://api.oleus.io/agent-audit/verify
| Field | Meaning |
|---|---|
| ok | True only if the chain is intact and the latest anchor (if any) matches. |
| rows_checked | Entries recomputed. |
| broken_at_seq, reason | First failure: hash_mismatch (edited row), seq_gap (deleted row) or a broken link. |
| anchor.status | ok, none, tail_truncated, head_mismatch, anchor_record_tampered or s3_unreachable. |
A hash chain alone cannot reveal deleted tail rows. Chain heads are therefore written daily to S3 Object Lock (compliance mode), and /verify compares the live chain with the latest anchor.
Auditor export
GET /agent-audit/export?control=CC6.1 (admin) returns CSV: timestamp, seq, member, action, soc2_controls, agent_id, trace_id, model, tool_calls, identity_source, agent_key_id, ai_act_articles, dora_articles, pii_detected_client_asserted, risk_score, approved_by, verification_status, hash, prev_hash. An auditor can recompute the chain from these columns.
Agent identity (per-agent keys)
By default agent_id is a label the client sends. For identity you can rely on, create a per-agent key (admin): POST /agent-keys {"agent_id":"support-agent"} returns a gak_… secret once; only its SHA-256 hash is stored. Use it in place of your member token in the SDK (wrap_openai(client, "gak_…")) or gateway (x-oleus-token).
With a key, the server fixes agent_id (a mismatched value is rejected with 403), records the key ID and identity_source in every log entry, and ties the key to the human who created it. Removing that member (for example via SCIM deprovisioning) disables their keys. DELETE /agent-keys/:id revokes immediately. Creation and revocation are written to the hash chain. Rotate by creating a new key, switching, then revoking the old one. PUT /agent-keys/enforce {"require_agent_keys":true} rejects member tokens on log, authorize and gateway calls. Keys cannot call admin endpoints.
Control mapping
| Action | Controls |
|---|---|
| agent_llm_call | CC7.2 |
| agent_tool_call, agent_tool_call_blocked | CC6.1, CC7.2 |
| agent_approval | CC6.1 |
| agent_policy_change | CC8.1 |
EU AI Act and DORA evidence
Each export row carries ai_act_articles and dora_articles. This is evidence that supports your obligations; it does not make a system compliant with the AI Act or DORA.
| Action | EU AI Act | DORA |
|---|---|---|
| agent_llm_call, agent_tool_call | Art. 12 (record-keeping) | Art. 12 (logging) |
| agent_tool_call_blocked | Art. 12, 14 (human oversight) | Art. 9, 12 |
| agent_approval | Art. 14 | Art. 9 (protection and prevention) |
| agent_policy_change | Art. 12 | Art. 9, 12 |
Art. 26 asks deployers of high-risk systems to keep automatically generated logs for at least six months; retention here is configurable from 365 days. Whether your system is high-risk, and which obligations apply, is for you and your counsel to decide. Oleus records hashes, not prompts, so it does not capture input data content that Art. 12 may require for some systems.
API reference
Base URL https://api.oleus.io. Authenticate with Authorization: Bearer <gretl token> (your member token from the dashboard). Errors return JSON {"error": ...}. Requests are rate limited per IP (HTTP 429 with Retry-After).
| Endpoint | Role | Purpose |
|---|---|---|
| POST /agent-audit/authorize | member | Pre-flight a tool call. Returns 200 allowed, or 202 with approval_request_id if the tool is sensitive and unapproved. |
| POST /agent-audit/log | member | Record an LLM call. Returns 200, or 202 if it contains an unapproved sensitive tool call. |
| GET /agent-audit | member | List entries. Query: page, limit (max 200), from, to, agent_id, tool_name, severity, pii_only. |
| GET /agent-audit/stats | member | Today's traces, tool calls, high-risk count, active agents, pending approvals. |
| GET /agent-audit/verify | member | Recompute the chain and check the latest S3 anchor. Query: from, to. |
| GET /agent-audit/approvals | admin | List approvals. Query: status (default pending). |
| POST /agent-audit/approvals/:id/approve | admin | Approve. Valid for 15 minutes, one agent and tool, single use. Optional body {"note"}. |
| POST /agent-audit/approvals/:id/deny | admin | Deny. |
| GET /agent-audit/export | admin | CSV with sequence numbers, hashes and previous hashes. Query: from, to, control (e.g. CC6.1). Max 10,000 rows. |
| POST /agent-keys | admin | Create a per-agent key. Body: agent_id, optional name. Secret returned once. |
| GET /agent-keys | admin | List keys (prefix, owner, last used, revoked). |
| DELETE /agent-keys/:id | admin | Revoke a key. |
| PUT /agent-keys/enforce | admin | Body: require_agent_keys. Reject member tokens for agent traffic. |
POST /agent-audit/log
{
"trace_id": "trace_123", "agent_id": "support-agent",
"provider": "openai", "model": "gpt-4o",
"prompt_hash": "sha256:...", "completion_hash": "sha256:...",
"prompt_tokens": 120, "completion_tokens": 48,
"tool_calls": [{"tool_name": "refund", "status": "requested"}],
"pii_detected_client": false, "pii_types": [],
"secrets_detected_client": false, "risk_score": 0,
"jit_request_id": "<approval uuid, optional>"
}
Required: trace_id, agent_id. A held call returns 202 {"status":"pending_approval","approval_request_id":"..."}. Re-send with jit_request_id after an admin approves.
Gateway endpoints
| Endpoint | Forwards to |
|---|---|
| POST /gateway/openai/v1/chat/completions | api.openai.com |
| POST /gateway/anthropic/v1/messages | api.anthropic.com |
Headers: x-oleus-token (required), x-oleus-agent (agent ID), x-oleus-approval-id (an approved request to consume). Send your provider key as usual (Authorization or x-api-key); it is forwarded and never stored. JSON and streaming (SSE) responses are supported; streams are buffered before returning. Errors:
403 oleus_approval_required: a sensitive tool call is held; body includesapproval_request_id.502 oleus_audit_unavailable: the call could not be recorded, so the response is withheld (fail closed).502 oleus_upstream_unreachable: the provider could not be reached.
Sensitive tools: send_email, send_slack, refund, delete, update_billing, execute_sql, create_user, transfer_money.
Self-hosting
The backend needs only Postgres (DATABASE_URL).
docker compose -f deploy/docker-compose.yml up -d OWNER_DATABASE_URL=postgres://postgres:postgres@localhost:5432/gretl deploy/migrate.sh
Run migrations as the database owner and run the backend as the gretl_app role. Set AUDIT_ANCHOR_BUCKET (an S3 bucket created with Object Lock enabled) to turn on anchoring. ClickHouse is optional and used only for telemetry, never for the chain or approvals.
Limits
- "Every call" holds only for traffic that goes through the SDK wrappers or the Oleus gateway (with direct provider egress blocked).
- By default the SDK fails open if the Oleus backend is unreachable, so an outage does not break your app. Calls made during an outage are not recorded. Pass
strict=Truetowrap_openai/wrap_anthropicto fail closed: a call that cannot be recorded raisesOleusAuditError. - PII detection is client-asserted.