Deterministic agent verification. Open-source test runner for coding agents and skills: run real workflows, capture evidence, assert on tools, commands, files, and answers. Get started · How it works · Documentation · Dashboard · Agent skills · Examples Assert on tool calls, commands, files, HTTP requests, transcripts, and final answers without requiring one model to judge another. Run the same scenario through Claude Code, OpenAI Codex, OpenCode, Pi, Cursor CLI, and Google Antigravity CLI to find behavior that varies between environments. Test multi-step tasks in fresh temporary work directories, repeat them to expose flaky behavior, and keep the evidence when something fails. Replace bare executable calls with scenario-scoped static, sequential, or handler responses using experimental CLI mocks. Provide HTTP MCP tool fixtures and check received calls with MCP mocks. Requirements Node.js 22+ and at least one supported harness installed, authenticated, and available on PATH: Claude Code (claude), Codex (codex), OpenCode 2.x (opencode), Pi (pi), Cursor CLI (cursor-agent), or Google Antigravity CLI 1.1.14+ (agy). Quick start npx dynobox@latest init # Claude Code (default) npx dynobox@latest init --harness codex # OpenAI Codex npx dynobox@latest init --harness opencode # OpenCode npx dynobox@latest init --harness pi # Pi npx dynobox@latest init --harness cursor # Cursor CLI npx dynobox@latest init --harness antigravity # Google Antigravity CLI npx dynobox run Run one init command for a harness that is installed and authenticated, then run the generated dyno. dynobox init creates a starter dyno in dynobox/example.dyno.mjs. dynobox run discovers *.dyno.{mjs,js,ts,mts,yaml,yml} files below the current directory and runs their scenarios against the configured harnesses. Only run dynos you trust. JavaScript and TypeScript configs are imported, and setup and verification commands execute on your machine. Temporary work directories separate job files, but they are not security sandboxes; processes can access the host according to their permissions. Writing a dyno A dyno combines the prompt, fixture setup, harnesses, and acceptance criteria in one TypeScript, JavaScript, or YAML file. Include only harnesses installed and authenticated in the current environment: # package.dyno.yaml name: package-script harnesses: - claude-code - codex - opencode - pi - cursor - antigravity scenarios: - name: detects the test script setup: - | cat > package.json <<'JSON' {"scripts":{"test":"vitest run"}} JSON prompt: >- Use cat package.json and tell me whether this project has a test script. assertions: - type: command.called executable: cat command: {args: [package.json]} - type: tool.notCalled tool: edit_file - type: artifact.contains path: package.json text: vitest run - type: finalMessage.contains text: test Run the test: npx dynobox run package.dyno.yaml The same shape works in TypeScript and JavaScript via @dynobox/sdk (defineDyno, tool, command, ...). See Config Authoring. Dynobox launches the selected harnesses in fresh temporary work directories, records the observable behavior, and evaluates each assertion against captured evidence. When a check fails, the output shows what was expected and what was observed. Assertions * HTTP capture observes child-process traffic that honors Dynobox's proxy and CA settings. Harness-native web tools and clients with independent networking may not be captured. Learn how HTTP capture works. See the assertion reference for every matcher and authoring option. Harnesses and iterations Override the configured harnesses from the command line: npx dynobox run --harness claude-code npx dynobox run --harness codex npx dynobox run --harness opencode npx dynobox run --harness pi npx dynobox run --harness cursor npx dynobox run --harness antigravity npx dynobox run --harness claude-code,codex,opencode,pi,cursor,antigravity Repeat every scenario and harness pair to measure pass rates: npx dynobox run \ --harness claude-code,codex,opencode,pi,cursor,antigravity \ --iterations 5 For each scenario, Dynobox runs the selected harness configurations in parallel. Repeated iterations for one harness configuration remain sequential, and the next scenario starts after every harness for the current scenario finishes. Repeated runs render compact pass-rate rows such as .FF.. while retaining the failed iteration evidence for diagnosis. Local runs and uploads Dynobox runs locally and works without an account. Use terminal output for development, --reporter json for automation, or authenticate with dynobox login and add --save-run to publish a compact run summary to the Dynobox dashboard. Set DYNOBOX_UPLOAD_URL to post the same payload to your own endpoint without sending Dynobox credentials. Saved-run data is length-capped but not redacted. It includes available Git identity and revision metadata. All jobs can include authored assertion data and matched evidence such as requested endpoint URLs, tool commands, and verification output. Failed jobs can additionally include command or harness diagnostics. Do not use --save-run when those values should not be shared. How Dynobox differs Dynobox is closer to unit testing than model-graded evals. It evaluates captured behavior—tool calls, shell commands, files, and HTTP requests—using explicit, repeatable assertions. Compared with one-off scripts, Dynobox provides normalized assertions, fresh temporary work directories, captured failure evidence, and iteration pass rates for measuring flaky behavior. Learn more Getting Started How It Works Config Authoring CLI Reference CI Integration Agent Skills Example dyno tests Packages Dynobox is early-access software. The CLI, SDK, and report formats may evolve before 1.0. Contributing See CONTRIBUTING.md to run the monorepo locally and propose a change. Dynobox is Apache-2.0 licensed.
AI NEWS JOURNAL
Hacker News Show HNAI NEWS / JOURNAL
Show HN: Dynobox – Integration tests for agents using CLI and MCP mocks
Article URL: https://github.com/dynobox/dynobox Comments URL: https://news.ycombinator.com/item?id=50565029 Points: 1 # Comments: 0
번역 서비스가 일시 중지되어 원문을 표시합니다.