Featured work
Three systems. Evaluation, runtime trust, and multi-agent analysis. Each treated as a product with a protocol, a test suite, and a failure mode.
Adapters
LangGraph · CrewAI · AutoGen
Golden YAML
Expected behavior
Production traces
Failure Memory in
Runner
Metrics + taxonomy
Baseline compare
CI exit codes
Dashboard
Case-level evidence
Open source · MIT · v0.3.0
CI for AI agents that turns production failures into minimized regression tests.
Sealed manifest
Canonical hash of exact effect
Action Passport
Intent → outcome, re-verified
Open source · MIT · v0.2.0 experimental
Seal the intended effect. Witness the actual outcome.
Orchestrator
Classify turn → route specialist
SQL
SELECT only
ML
AutoML
Stats
Tests
RAG
Memory
Open source · MIT · live demo
Upload data. Ask in English. Get SQL, AutoML, stats, forecasts, and a living report.
Engineering philosophy
I do not ship agents that merely look intelligent. I ship systems that can be measured, gated, and proven.
Read the manifestoReliability is the product. Latency, cost, and fluency are constraints. If the system cannot be trusted under retry, timeout, and partial failure, it is not ready to act.
Golden cases, baselines, and CI gates come before the demo reel. AgentEval exists because I refuse to treat probabilistic software as untestable.
The strongest credibility is a public test suite, a PyPI package, and a threat model written in the same voice as the code. Claims without artifacts are marketing.
Permission to call a tool is not permission to change the world. Safety lives in sealed intents, TOCTOU checks, exactly-once execution, and independent verification.
Coverage gates, strict types, crash recovery, and honest limitations. If it is experimental, the README says so. If it is irreversible, the protocol says so.
A demo is a path. A system is a state machine with invariants. I build the latter: evaluation loops, trust protocols, and agent runtimes that survive contact with reality.
Open source
Nishant Tyagi
@nishanttyagi28
Python Backend Developer | FastAPI, PostgreSQL, Docker | GenAI (RAG, LLM eval) & Data Analytics.
Contribution activity
Updated from GitHub hourly
Writing
Path
2015 — 2018
University of Delhi
Trained to reason about incentives, trade-offs, and systems that fail at the margin. That habit never left: every agent action has an expected effect, an actual effect, and a cost of being wrong.
MCA
Graduate study
Formal shift from economic systems to computational ones. Coursework and independent builds around machine learning, data systems, and the gap between model output and production behavior.
Independent
Product and data systems
Shipped analysis pipelines, RAG backends, and client-facing agent tools. Learned the unglamorous parts: evaluation, guardrails, and the difference between a notebook and a system someone else has to operate.
2026
Open source · PyPI
Released a git-native evaluation harness with Failure Memory. 1115 deterministic tests, framework adapters, and a CI gate that turns production failures into minimized regression cases.
2026
Open source · PyPI
Shipped runtime trust infrastructure for consequential agent actions: cryptographic sealing, exactly-once execution, independent verification, and 74 tested security invariants.
2026
Live product demo
Launched a multi-agent analysis system with text-to-SQL, AutoML, statistical testing, forecasting, and ChromaDB memory behind a single chat.
Now
Evaluation, safety, runtime trust
Building the layer most agent products skip: evaluation gates, effect verification, and open-source infrastructure that can be inspected, tested, and deployed.
Contact
Evaluation harnesses, runtime trust protocols, and agent infrastructure. If you are hiring for AI reliability or want to collaborate on open source, start here.