02 · Runtime trust infrastructure
KarmaSakshi Protocol
Seal the intended effect. Witness the actual outcome.
- Protocol tests
- 1049
- Security invariants
- 74
- Buyer acceptance
- 25/25
- Coverage gate
- 90.5%
The story
Karma means action. Sakshi means witness. I built KarmaSakshi because evaluation and runtime trust are different jobs. AgentEval asks whether the agent behaved during CI. KarmaSakshi asks whether the approved real-world effect actually happened, once, and can be proven. IAM, OAuth, and policy gates sit upstream. This protocol sits after them.
Problem
Tool permissions answer one question: may this agent call this tool? They do not prove that the exact effect a human approved is the exact effect that executed. Retries, stale state, and post-approval tampering are enough to move money twice or to the wrong recipient.
Why it matters
As agents gain the ability to refund, transfer, email, and mutate records, allow/deny becomes a liability. Production systems need a protocol that binds approval to one canonical effect, revalidates at commit time, and independently observes the world afterward.
Architecture
PROPOSE → PREPARE → SEAL → AUTHORIZE → COMMIT → VERIFY → PROVE. The agent never holds a signing key and never issues its own grant. The engine is the only component that validates authorization and decides whether an adapter commit() runs. Adapters perform and independently re-observe external state. The audit journal is hash-chained. The Action Passport re-verifies seal, grant, and chain at generation time.
Protocol
Allowed is not verified.
01
PROPOSE
02
PREPARE
03
SEAL
04
AUTHORIZE
05
COMMIT
06
VERIFY
07
PROVE
Agent / LLM
Produces a request. Never holds a signing key.
Engine
Validates grant. Only path to commit().
Adapters
Perform + independently re-observe.
External system
Ledger, mail, row. Source of truth.
Untrusted
Agent, model, retry loops, tool-calling runtime
Trust boundary
Seal, grant, TOCTOU revalidation, reservation slot, audit chain
Observed world
Independent verify() against the system of record
| Threat | Control | Result |
|---|---|---|
| Post-approval tamper | Manifest hash bound into grant | Blocked |
| Stale world / TOCTOU | Commit-time precondition recheck | Blocked |
| Duplicate refund retry | Exactly-once reservation slot | Blocked |
| Agent-issued grant | Invariant 30 — human/service only | Blocked |
| Success lie from provider | Independent re-observation | Detected |
| Ambiguous crash | Recover by observing, never blind retry | Contained |
Technical decisions
Bind approval to a sealed manifest, not a tool name
A grant is valid for one canonical Effect Manifest hash. Changing amount, recipient, or preconditions after approval is tampering, not a new request.
Commit-time TOCTOU revalidation
Preconditions are re-checked immediately before the effect. If the world drifted since approval, the commit fails closed with StaleManifestError.
Exactly-once successful-effect semantics
A slot is reserved before the adapter runs and permanently consumed only on success. Concurrent retries race for one slot. Ambiguous crashes are recovered by re-observation, never by blind retry.
Independent verification
A provider success response is not proof. adapter.verify() re-queries the system of record. Mismatches are recorded, not hidden.
Challenges
Crash between effect and local finalization
If the process dies after the payment landed but before the store commits, retry is dangerous. Recovery re-observes external state before considering another attempt.
Honest compensation
Irreversible effects are never described as reversible. Compensation is best-effort and says so in the protocol, not in marketing copy.
Trust boundary discipline
The model produces a request. Humans and services issue grants. The engine decides. Adapters do not authorize. Invariant 30 is enforced in code.
Outcomes
- 01
v0.2.0 experimental protocol with Milestone A evaluation-ready self-hosted software.
- 02
1049 tests passed, 90.50% coverage, mypy strict clean, ruff/bandit/pip-audit clean.
- 03
74 security invariants implemented and mapped to tests. 25/25 buyer acceptance checks.
- 04
Gateway, typed SDK, authenticated Control Center, Docker Compose evaluation, LangGraph integration, AgentEval bridge.