Skip to content
NT

02 · Runtime trust infrastructure

KarmaSakshi Protocol

Seal the intended effect. Witness the actual outcome.

Protocol tests
1049
Security invariants
74
Buyer acceptance
25/25
Coverage gate
90.5%

The story

Karma means action. Sakshi means witness. I built KarmaSakshi because evaluation and runtime trust are different jobs. AgentEval asks whether the agent behaved during CI. KarmaSakshi asks whether the approved real-world effect actually happened, once, and can be proven. IAM, OAuth, and policy gates sit upstream. This protocol sits after them.

Problem

Tool permissions answer one question: may this agent call this tool? They do not prove that the exact effect a human approved is the exact effect that executed. Retries, stale state, and post-approval tampering are enough to move money twice or to the wrong recipient.

Why it matters

As agents gain the ability to refund, transfer, email, and mutate records, allow/deny becomes a liability. Production systems need a protocol that binds approval to one canonical effect, revalidates at commit time, and independently observes the world afterward.

Architecture

PROPOSE → PREPARE → SEAL → AUTHORIZE → COMMIT → VERIFY → PROVE. The agent never holds a signing key and never issues its own grant. The engine is the only component that validates authorization and decides whether an adapter commit() runs. Adapters perform and independently re-observe external state. The audit journal is hash-chained. The Action Passport re-verifies seal, grant, and chain at generation time.

Protocol

Allowed is not verified.

LifecyclePROPOSE → PROVE
  1. 01

    PROPOSE

  2. 02

    PREPARE

  3. 03

    SEAL

  4. 04

    AUTHORIZE

  5. 05

    COMMIT

  6. 06

    VERIFY

  7. 07

    PROVE

Security architectureWho may commit

Agent / LLM

Produces a request. Never holds a signing key.

Engine

Validates grant. Only path to commit().

Adapters

Perform + independently re-observe.

External system

Ledger, mail, row. Source of truth.

Trust boundariesInvariant 30

Untrusted

Agent, model, retry loops, tool-calling runtime

Trust boundary

Seal, grant, TOCTOU revalidation, reservation slot, audit chain

Observed world

Independent verify() against the system of record

Threat modelFail closed
Threat model controls and outcomes
ThreatControlResult
Post-approval tamperManifest hash bound into grantBlocked
Stale world / TOCTOUCommit-time precondition recheckBlocked
Duplicate refund retryExactly-once reservation slotBlocked
Agent-issued grantInvariant 30 — human/service onlyBlocked
Success lie from providerIndependent re-observationDetected
Ambiguous crashRecover by observing, never blind retryContained

Technical decisions

Bind approval to a sealed manifest, not a tool name

A grant is valid for one canonical Effect Manifest hash. Changing amount, recipient, or preconditions after approval is tampering, not a new request.

Commit-time TOCTOU revalidation

Preconditions are re-checked immediately before the effect. If the world drifted since approval, the commit fails closed with StaleManifestError.

Exactly-once successful-effect semantics

A slot is reserved before the adapter runs and permanently consumed only on success. Concurrent retries race for one slot. Ambiguous crashes are recovered by re-observation, never by blind retry.

Independent verification

A provider success response is not proof. adapter.verify() re-queries the system of record. Mismatches are recorded, not hidden.

Challenges

Crash between effect and local finalization

If the process dies after the payment landed but before the store commits, retry is dangerous. Recovery re-observes external state before considering another attempt.

Honest compensation

Irreversible effects are never described as reversible. Compensation is best-effort and says so in the protocol, not in marketing copy.

Trust boundary discipline

The model produces a request. Humans and services issue grants. The engine decides. Adapters do not authorize. Invariant 30 is enforced in code.

Outcomes

  1. 01

    v0.2.0 experimental protocol with Milestone A evaluation-ready self-hosted software.

  2. 02

    1049 tests passed, 90.50% coverage, mypy strict clean, ruff/bandit/pip-audit clean.

  3. 03

    74 security invariants implemented and mapped to tests. 25/25 buyer acceptance checks.

  4. 04

    Gateway, typed SDK, authenticated Control Center, Docker Compose evaluation, LangGraph integration, AgentEval bridge.

← All work