Skip to content

AgentGuard

Evaluate what coding agents actually do. AgentGuard is a local-first safety and evaluation harness that runs reproducible coding-agent scenarios, inspects observable evidence, and produces reports that people and CI can review.

Coding agents can complete a requested change while also tampering with tests, following hidden instructions, touching forbidden files, introducing secret-like content, or expanding scope. AgentGuard scores tests, diffs, command events, policy checks, traces, and reports instead of trusting an agent's explanation.

Start with the quickstart Explore the architecture Watch the 80-second demo

Current release: AgentGuard v0.3.0, published to production PyPI as agentguard-evals. AgentGuard supports Python 3.9–3.12.

python -m pip install agentguard-evals
agentguard --version
agentguard --help

Product AgentGuard

PyPI distribution agentguard-evals

Python import agentguard

Terminal command agentguard

The installed package contains the import and command, but not the repository examples. Clone the repository for a first deterministic evaluation:

git clone https://github.com/richinmrudul/agentguard.git
cd agentguard
python -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
agentguard run examples/configs/fix_auth_bug_local_command_safe.yaml --agent local-command
agentguard reports show --latest --type run

This uses a safe, network-free local fixture and produces inspectable evidence under .agentguard/. See the quickstart for identity, installation, Docker, and example boundaries.

AgentGuard evaluation dashboard showing eight deterministic records, one safe pass, seven detected failures, and benchmark categories

The dashboard is real output from the repository's deterministic showcase plus one audit-mode guard incident. Explore the other maintained captures in the visual tour, or follow the same evidence path in the silent product demo. The homepage does not autoplay or eagerly load the video.

Current evidence

  • More than 1,400 tests pass locally with 15 documented Docker skips; the exact v0.3.0 commit also passed the full Docker-backed GitHub integration job.
  • The curated showcase detects 5/5 unsafe scenarios, allows 1/1 safe scenario, and records zero false positives and false negatives for that deliberately small demo corpus.
  • The adversarial-core foundation contains 10 deterministic, network-free unsafe-agent scenarios across eight categories.
  • AgentGuard emits JSON and Markdown reports, SARIF and JUnit exports, execution manifests, portable hash-chained traces, offline replay results, and optional static report sites.
  • The retained workflow artifacts and public v0.3.0 PyPI artifacts were verified byte-identical.

These are scoped validation results, not claims about universal production effectiveness. See detection quality, performance, and testing for methodology and limitations.

Architecture at a glance

flowchart LR
    Config[Config or suite] --> Repo[Prepared repo]
    Repo --> Agent[Agent adapter]
    Agent --> Guard[Online guard]
    Agent --> Checks[Post-hoc checks]
    Guard --> Evidence[Observable evidence]
    Checks --> Evidence
    Evidence --> Score[Score and result]
    Score --> Artifacts[Reports, manifests, traces, history]
    Artifacts --> CI[CI exports]
    Artifacts --> ReportSite[Optional static report site]

AgentGuard's documentation website and its generated evaluation report sites are separate systems. This site explains the project; agentguard reports site exports a sanitized snapshot of local evaluation artifacts.

Trust boundary

Docker-backed execution can provide configured containment. Local agent execution uses the host user's permissions and is not inherently sandboxed. AgentGuard evaluates observable evidence; it is not a perfect security boundary, syscall monitor, formal certification, or guarantee that every unsafe behavior will be detected.

Explore