api-relay-audit

Overview of api-relay-audit

What it is

API Relay Audit is a local security audit tool for AI API relays and LLM proxies. It checks observable relay behavior and generates a Markdown report covering prompt injection, model substitution, tool-call rewriting, SSE anomalies, error leakage, and Web3 wallet risks. The tool is informed by the paper Your Agent Is Mine and is designed to run independently without sending data to external services.

Key features

• Detects hidden prompt injection, instruction override, and context truncation.
• Checks response integrity for changes to pinned package-command text, error-response leakage, and SSE stream anomalies.
• Provides per-step evidence with LOW / MEDIUM / HIGH risk summaries; inconclusive probes remain visible.
• Supports Web3 wallet safety probes for transfer guidance, signed-transaction refusal, and private-key refusal.
• Outputs Markdown or JSON reports for local automation.
• Runs as a standalone Python script using only the standard library and curl.

Use cases

• You use a third-party AI API relay, mirror, gateway, or LLM proxy.
• You want to check whether a proxy injects prompts, swaps models, truncates context, or rewrites tool output.
• You are testing relay behavior before production traffic, coding-agent automation, package-install suggestions, or wallet-related actions.
• You need a local, repeatable audit report instead of a web tool that asks for your API key.

Getting started

bash AUDIT_SCRIPT_REF=v2.5.0 curl -fsSL 'https://raw.githubusercontent.com/toby-bridges/api-relay-audit/${AUDIT_SCRIPT_REF}/audit.py' -o audit.py python audit.py --key --url --output report.md # Web3 / wallet users python audit.py --key --url --profile web3 --output report.md

License and pricing

Licensed under AGPL-3.0. Open source.

Based on the project's public README and repository metadata.