Security testing for AI agents is easier to operationalize when it behaves like ordinary software testing: install a package, point it at a target, run a known corpus, preserve the raw result, and fail or review the build when a regression appears. safelabs-eval is an open-source Python framework built around that workflow.
The current project ships 300 adversarial prompts across ten internally defined ASI categories. Each category has 30 prompts, split evenly across three author-defined difficulty tiers. The repository is careful about terminology: the category structure is OWASP-inspired, but it is not the official OWASP Top 10 for Agentic Applications 2026 and is not currently mapped to that taxonomy.
1. Install the core package
The core package supports HTTP targets and Python callables without requiring a framework-specific dependency.
pip install safelabs-eval
2. Test a supported HTTP endpoint
For a service that accepts a prompt over HTTP, the CLI can run one category or the full corpus. The exact request shape still has to match the endpoint contract; the framework is not a universal API auto-discovery layer.
safelabs run --target http://localhost:8000/chat --category ASI01
safelabs run --target http://localhost:8000/chat --category all --output json
3. Wrap a Python callable
For local development, notebooks, or custom orchestration code, wrap a function that accepts a prompt and returns a response.
import asyncio
from safelabs import run_eval
def my_agent(prompt: str) -> str:
return your_agent.run(prompt)
result = asyncio.run(run_eval(my_agent, categories=["ASI01", "ASI06"]))
result.summary()
4. Use a framework adapter when it helps
The current repository includes optional adapters for LangChain, CrewAI, AutoGen/ag2, LlamaIndex, OpenAI Agents SDK, Google ADK, and Semantic Kernel. Install only the extra required for the application under test. The core package remains usable without those dependencies.
For example, a LangChain Runnable can be wrapped with LangChainAdapter and passed to run_eval. A Google ADK agent can be wrapped with GoogleADKAdapter, while Semantic Kernel agents use SemanticKernelAdapter.
5. Understand the verdicts before gating CI
safelabs-eval exposes four verdicts: VULNERABLE, FAIL, UNCERTAIN, and PASS. The current definitions are deliberately conservative. VULNERABLE means the detector found apparent attack compliance; FAIL indicates signs of weakness without apparent full compliance; UNCERTAIN means the detector evidence was ambiguous; PASS means no unsafe compliance was detected in that response.
The confidence value printed by the CLI is not a calibrated probability. It is a heuristic evidence score from the pattern detector. A 70% confidence label should therefore not be interpreted as a 70% probability that the verdict is objectively correct.
6. Preserve enough context to reproduce the result
The project’s detector-reliability research found that raw verdict counts can be misleading when detector logic changes. For CI or published benchmarking, preserve the raw model response before scoring, along with the model identifier, framework version, safelabs-eval version, prompt-corpus content version, and repository commit.
The current prompt library reports content version 1.13.0 and schema version 1.1.0 in the repository dataset card. The content version changes as prompt batches are added, so a commit hash is the safest way to pin the exact corpus used in a test.
7. Know what the 300-prompt corpus does and does not mean
SafeAgent-300 is balanced for coverage: 30 prompts per category and 10 prompts in each of three difficulty tiers inside every category. The tiers are an author-defined attack-sophistication rubric, not an empirically calibrated measure of how often models fail.
The corpus is English-only and single-turn. Some Behavioral Drift prompts describe multi-turn scenarios in the text, but the framework does not execute a persistent multi-turn conversation for those cases. Coverage is breadth-oriented rather than exhaustive.
8. Use metadata for analysis, not for overclaiming
Prompt entries now include provenance metadata and MITRE ATLAS technique identifiers where the project has a reasonable mapping. Two categories use UNMAPPED because no ATLAS technique cleanly fits. The dataset card documents several nearest-fit mappings and warns that they are not authoritative ATLAS classifications.
That transparency is useful in developer tooling: metadata can improve filtering and reporting without pretending the taxonomy is more standardized than it is.
A minimal CI policy
- Run a pinned prompt subset on every agent or model change.
- Persist JSON output plus raw model responses as build artifacts.
- Block releases on confirmed VULNERABLE findings.
- Route UNCERTAIN and high-severity FAIL results to human review rather than automatically treating them as safe.
- Re-run the same corpus after remediation to detect regression.
- Record the exact detector and prompt-corpus versions in the report.
Why this workflow is more useful than a one-off demo
Agent security is not static. A model upgrade, prompt rewrite, tool-permission change, framework migration, or memory feature can change behavior without breaking ordinary functional tests. Reproducible adversarial tests provide a regression layer for those changes.
The goal is not to produce a single “safe” score. It is to make failures observable, reviewable, and repeatable enough that engineering teams can treat agent security as part of the software lifecycle.
