Skip to main content
The banner shows the boxd.sh and Harbor logos, with Harbor described as the agent evaluation framework. Harbor is a framework for evaluating and optimizing AI agents and models: it runs agents like Claude Code, Codex, or OpenHands against benchmarks like Terminal-Bench and SWE-Bench across many parallel sandboxes. The boxd environment provider gives every trial its own isolated KVM microVM with a full Docker daemon, so Dockerfile and multi-container docker-compose.yaml tasks run unmodified:
For benchmarks, start from a snapshot: prepare a machine, save a snapshot of it, and create every sandbox from it with --ek from_snapshot=<snapshot>. This shortens startup, because each trial gets a machine whose Docker cache already holds the task image instead of building it from scratch. The provider is awaiting merge upstream. Until it merges, install Harbor from the pull request’s branch and everything above works.