> ## Documentation Index
> Fetch the complete documentation index at: https://docs.boxd.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# Inspect AI

> Run every Inspect AI sample on its own boxd machine. Tasks written for Inspect's Docker sandbox run unchanged, with Docker inside the machine.

![The Inspect wordmark and the AI Security Institute logo inside the boxd card.](https://mintcdn.com/azin/GUvqATY0IhPdUBtf/images/boxd-x-inspect-ai.png?fit=max\&auto=format\&n=GUvqATY0IhPdUBtf\&q=85\&s=9978662fc56b7aa69d76e8fb321d0aa7)

[Inspect AI](https://inspect.aisi.org.uk) is the open-source framework for large language model evaluations from the UK AI Security Institute. An eval runs its samples in sandboxes, where the model's tools run commands and read and write files.

[`boxd-inspect`](https://github.com/boxd-sh/boxd-inspect) adds the sandbox type `boxd`. Each sample runs on its own boxd machine, a full Ubuntu 24.04 VM, and the machine is deleted when the sample ends. Tasks with a `compose.yaml` or `Dockerfile` run unchanged: Docker runs them inside the machine. Inspect added it to the Sandboxes section of its extensions list ([pull request](https://github.com/UKGovernmentBEIS/inspect_ai/pull/6510)).

## Set up

**1. Install the package**

```bash theme={"theme":"github-dark"}
pip install boxd-inspect
```

It needs Python 3.11 or later. Installing it registers the sandbox type `boxd` with Inspect, so there is nothing to import in your task.

**2. Create an API key**

```bash theme={"theme":"github-dark"}
boxd auth keys create inspect --org <your-org>
export BOXD_API_KEY=bxd_...
```

Machines are created and billed in the key's organization. The key is read from the environment only, so it never ends up in the eval log.

**3. Run a task on boxd**

```python kernel.py theme={"theme":"github-dark"}
from inspect_ai import Task, task
from inspect_ai.dataset import Sample
from inspect_ai.scorer import includes
from inspect_ai.solver import generate, use_tools
from inspect_ai.tool import bash


@task
def kernel():
    return Task(
        dataset=[
            Sample(
                input="Run `uname -s` with the bash tool and tell me its output.",
                target="Linux",
            )
        ],
        solver=[use_tools(bash(timeout=60)), generate()],
        scorer=includes(),
        sandbox="boxd",
    )
```

```bash theme={"theme":"github-dark"}
pip install anthropic
export ANTHROPIC_API_KEY=...
inspect eval kernel.py --model anthropic/claude-haiku-5-5
```

```text theme={"theme":"github-dark"}
kernel (1 sample): anthropic/claude-haiku-5-5
total time:                  0:00:03
includes
accuracy  1.000
```

Any model Inspect supports works.

## Run a Docker task on boxd

A task written for Inspect's Docker sandbox runs on boxd when you override the sandbox type. Its own `Dockerfile` or `compose.yaml` is used:

```bash theme={"theme":"github-dark"}
inspect eval my_task.py --sandbox boxd --model anthropic/claude-haiku-5-5
```

You can also name the file in the task: `sandbox=("boxd", "compose.yaml")`. The compose file's directory goes to the machine, so `build:` contexts next to it work. Several services share one machine, and the service named `default` (or marked `x-default: true`) is the default sandbox, as in Docker.

Without a compose file, commands run on the machine itself as the `boxd` user, with passwordless `sudo`, Docker, Python, Node, Go and git, in `/home/boxd`.

## Configuration

Pass a `BoxdSandboxEnvironmentConfig`, or a JSON file with the same fields as `sandbox=("boxd", "boxd.json")`:

```python theme={"theme":"github-dark"}
from inspect_ai.util import SandboxEnvironmentSpec

from boxd_inspect import BoxdSandboxEnvironmentConfig

sandbox = SandboxEnvironmentSpec(
    "boxd",
    BoxdSandboxEnvironmentConfig(region="fra", vcpu=4, delete_after=3600),
)
```

| Field | Default | What it does |
| - | - | - |
| `compose` | none | A `compose.yaml` or `Dockerfile` to run with Docker inside the machine. |
| `region` | `BOXD_REGION`, then your account's | The [region](https://docs.boxd.sh/guides/regions.md), `fra` or `nyc`. |
| `org` | `BOXD_ORG`, then the key's | The organization the machines belong to. |
| `vcpu`, `memory` | the organization's default size | The [machine size](https://docs.boxd.sh/guides/resources.md#machine-sizes): 1, 2, 4 or 8 vCPUs with 4 GiB of memory each. |
| `from_snapshot` | none | Start each sample from this [snapshot](https://docs.boxd.sh/guides/snapshots.md), for example one with your images already pulled. |
| `isolated` | `true` | The agent's commands get no route to your other machines and no boxd credentials. Outbound internet still works. Turn it off only for code you trust. |
| `working_dir` | `/home/boxd`, or the container's | Where commands run. |
| `name_prefix` | `inspect` | Machine names are `<prefix>-<task>-<sample>-<random>`. |
| `labels` | none | Extra labels on every machine. Each machine also gets `inspect-sandbox=boxd`. |
| `delete_after` | none | Seconds after creation at which boxd [deletes the machine](https://docs.boxd.sh/guides/delete-after.md), whatever it is doing. Keep it above your longest sample. |
| `ready_timeout` | `180` | Seconds to wait for a new machine. |
| `compose_timeout` | `900` | Seconds `docker compose up` may take, pulls and builds included. |

Limit how many machines run at once with Inspect's `--max-sandboxes`. Your organization's machine quota is the upper bound, and `boxd manage billing` shows it.

## Clean up

A sample's machine is deleted when the sample ends, also when it fails or is cancelled. Any machine left over is deleted by id when the eval ends. With `--no-sandbox-cleanup` the machines stay, and Inspect prints their ids:

```text theme={"theme":"github-dark"}
boxd machines left running for inspection: <machine-id>
Delete them with: inspect sandbox cleanup boxd <id>
```

Delete one when you are done:

```bash theme={"theme":"github-dark"}
inspect sandbox cleanup boxd <machine-id>
```

If the eval process is killed, its machines keep running. Set `delete_after` so boxd deletes them for you. `inspect sandbox cleanup boxd` without an id deletes every machine labelled `inspect-sandbox=boxd` in the organization that is not running, and keeps running ones because they can belong to someone else's eval.

## Limits

* **One machine per sample,** shared by all services of its compose file.
* **Fresh machines:** compose images are pulled for every sample. For big images, prepare a machine once, save a snapshot and pass `from_snapshot`.
* **Compose directory:** up to 512 MiB, sent to the machine as one archive.
* **Command output:** 10 MiB per stream, and `read_file` up to 100 MiB, the same limits as every Inspect sandbox.

## Read more

* [`boxd-inspect` on GitHub](https://github.com/boxd-sh/boxd-inspect): errors, limits, how it works, and four examples
* [`boxd-inspect` on PyPI](https://pypi.org/project/boxd-inspect/)
* [Sandboxing](https://inspect.aisi.org.uk/sandboxing.html) in Inspect's docs
* [Sandboxes on boxd](https://docs.boxd.sh/use-cases/sandboxes.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.