> ## Documentation Index
> Fetch the complete documentation index at: https://docs.boxd.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# CI runners

> Build your own CI on boxd: every push and pull request runs on fresh machines created from a snapshot of main's last green run.

CI as one [automation](/guides/automations/scripts) and a config file: every push and pull request runs on fresh machines created from a snapshot of main's last green run, and reports a GitHub check. Both files are [below](#the-complete-files).

<Frame caption="A push to main: nine 2 vCPU machines, green in under two and a half minutes, saved as the next snapshot. Waits are cut short.">
  <video autoPlay muted loop playsInline src="https://mintcdn.com/azin/toBfp1IaOCRwpFnS/videos/ci-run.mp4?fit=max&auto=format&n=toBfp1IaOCRwpFnS&q=85&s=c993ccedc6adc187a982e2929c3b9407" data-path="videos/ci-run.mp4" />
</Frame>

## Why run CI on boxd

* **The machine is the cache.** Jobs start from a [snapshot](/guides/snapshots) with the repo, dependencies, build caches and Docker layers in place.
* **No idle runners.** Machines exist only while a job runs, metered per second.
* **Parallel by default.** Each test shard gets its own machine, with no setup to repeat.
* **Isolated.** Job machines hold no credentials, so pull request code can't reach your organization.
* **A real machine.** Docker, systemd and Postgres run normally.

## What you build

<img src="https://mintcdn.com/azin/nQ1a0C7OOgyLkkJH/images/ci-runners-architecture.svg?fit=max&auto=format&n=nQ1a0C7OOgyLkkJH&q=85&s=fa6f43cd83c6366cadf94796bc9b3b66" alt="A push or pull request on GitHub starts a run. boxd creates the job machines, one for the checks and one per test shard, all at once from the snapshot of main's last green run. They post a GitHub check on the commit, and a green run on main is saved as the next snapshot." width="820" height="352" data-path="images/ci-runners-architecture.svg" />

You need the [boxd GitHub App](/guides/integrations/connections#github-app) on the repository, and one machine to run the automation on. That machine can sleep: an event wakes it.

## 1. Snapshot your repo

GitHub triggers the run and shows the result. boxd runs it, with no workflow file and no Actions minutes.

Set up a machine the way a job should find it, then save it.

```bash theme={"theme":"github-dark"}
boxd new web-ci
boxd connect web-ci
```

Inside the machine:

```bash theme={"theme":"github-dark"}
mkdir -p ~/work && cd ~/work
gh repo clone acme/web && cd web
yarn install --frozen-lockfile
yarn lint --cache && yarn typecheck --incremental     # builds the caches
exit
```

Back on your laptop:

```bash theme={"theme":"github-dark"}
boxd snapshots save web-ci web-main
```

You do this once. After that, green runs on `main` save the next version ([step 6](#6-promote-a-green-main)).

## 2. Describe your CI

One entry per repository:

```json ci.json theme={"theme":"github-dark"}
{
  "acme/web": {
    "snapshot": "web-main",
    "workdir": "/home/boxd/work/web",
    "on": { "push": ["main"], "pull_request": true },
    "setup": [
      "git diff --quiet \"$BASE\" HEAD -- yarn.lock || yarn install --frozen-lockfile"
    ],
    "checks": [
      "yarn lint --cache",
      "yarn typecheck --incremental"
    ],
    "fanout": { "command": "yarn test --shard={i}/{n}", "n": 8 },
    "promote": "main"
  }
}
```

| Key | What it does |
| - | - |
| `setup` | Runs on every machine after checkout. `$BASE` is the snapshot's commit, so install is skipped when the lockfile is unchanged. |
| `checks` | Run in order on the first machine. |
| `fanout` | One more machine per shard, each running `command` with `{i}` and `{n}` filled in. |
| `promote` | The branch whose green runs become the next snapshot. |

## 3. Start a run on every push and pull request

Subscribe to events through the GitHub App. There is no webhook to host:

```typescript ci.run.ts theme={"theme":"github-dark"}
import { boxd, githubApp } from "@boxd/run";
import config from "./ci.json";

githubApp.on("push", { repo: "acme/web" }, async (e) => {
  const branch = e.ref.replace("refs/heads/", "");
  if (branch === "main") await ci(e.after, branch);
});

githubApp.on("pull_request", { repo: "acme/web" }, async (e) => {
  if (["opened", "synchronize", "reopened"].includes(e.action))
    await ci(e.pull_request.head.sha, e.pull_request.head.ref);
});
```

## 4. Create a machine per job

Create an isolated machine from the snapshot and check out the commit. The GitHub App token is short-lived and exists only in the fetch command's environment:

```typescript theme={"theme":"github-dark"}
async function prepare(name: string, sha: string) {
  const m = await boxd.machines.create({ name, fromSnapshot: snapshot, isolated: true });
  await boxd.machines.waitUntilReady(m.id);
  const token = await githubApp.getToken();
  await boxd.machines.exec(m.id, {
    command: ["bash", "-lc", `cd ${workdir} && git -c http.extraheader="AUTHORIZATION: basic $AUTH" fetch -q origin ${sha} && git checkout -q -f ${sha}`],
    env: { AUTH: btoa(`x-access-token:${token}`) },
  });
  for (const step of repo.setup) await run(m, step);
  return m;
}
```

The snapshot already holds main's checkout, so the fetch brings only the new commits.

## 5. Run everything at once

One machine for the checks and one per shard, all created together:

```typescript theme={"theme":"github-dark"}
async function ci(sha: string, branch: string) {
  const { n, command } = repo.fanout;
  const shards = Array.from({ length: n }, async (_, i) => {
    const m = await prepare(`ci-${sha.slice(0, 7)}-s${i + 1}`, sha);
    return run(m, command.replace("{i}", `${i + 1}`).replace("{n}", `${n}`));
  });

  const main = await prepare(`ci-${sha.slice(0, 7)}`, sha);
  let ok = true;
  for (const step of repo.checks) ok &&= await run(main, step);
  ok &&= (await Promise.all(shards)).every(Boolean);

  await report(sha, ok);                 // a GitHub check, next step
  if (ok && branch === repo.promote) await promote(main);
  // every machine except a promoted one is deleted here
}
```

<Frame caption="A pull request: checks and 8 test shards on nine 2 vCPU machines, green in 145 seconds. Waits are cut short.">
  <video autoPlay muted loop playsInline src="https://mintcdn.com/azin/toBfp1IaOCRwpFnS/videos/ci-pr.mp4?fit=max&auto=format&n=toBfp1IaOCRwpFnS&q=85&s=c65ea8a351abf74160c9235236dba85f" data-path="videos/ci-pr.mp4" />
</Frame>

Report the result as a check on the commit:

```typescript theme={"theme":"github-dark"}
const gh = await githubApp.client();
await gh.rest.checks.create({
  owner, repo: name, name: "boxd-ci", head_sha: sha,
  status: "completed", conclusion: ok ? "success" : "failure",
  output: { title: ok ? "green on boxd" : "failed", summary: stepTable },
});
```

## 6. Promote a green main

After a green run on `main`, the machine holds the state the next job should start from. Save it once the result is reported, so nobody waits on the save:

```typescript theme={"theme":"github-dark"}
async function promote(machine) {
  await boxd.snapshots.create(machine.id, next);

  // create() returns while the copy is still running, and the copy reads from
  // the live machine: keep it until the snapshot is ready.
  let snap = await boxd.snapshots.get(next);
  while (snap.status !== "ready") {
    await sleep(1000);
    snap = await boxd.snapshots.get(next);
  }
  snapshot = next;                   // the next run starts from it
  await boxd.machines.delete(machine.id);
}
```

Pull requests never write to the snapshot.

<Note>
  A snapshot can't be used to create machines while it saves a new version. Keep two names (`web-main` and `web-main-b`), save into the idle one, then switch.
</Note>

## 7. Turn it on

```bash theme={"theme":"github-dark"}
run ci.run.ts
```

<Frame caption="The config, then the automation started with one command.">
  <video autoPlay muted loop playsInline src="https://mintcdn.com/azin/toBfp1IaOCRwpFnS/videos/ci-config.mp4?fit=max&auto=format&n=toBfp1IaOCRwpFnS&q=85&s=9e722c5d5435f4921fa69943066286b6" data-path="videos/ci-config.mp4" />
</Frame>

```text theme={"theme":"github-dark"}
boxd-ci watching acme/web
2f5c7: running in the background — `run logs 2f5c7` to follow
```

Push a commit and follow the run with `run logs 2f5c7 -f`:

```text theme={"theme":"github-dark"}
▶ push to main · f9add68 · 9 machines from web-main
   4.2s  ✓ 9 machines restored from the snapshot, each booted in 4–290ms
   8.5s  ✓ lint  (4.3s)
  14.3s  ✓ typecheck  (4.2s)
  79.5s  ✓ shard 5/8  (74.2s)
  ...
 143.0s  ✓ shard 6/8  (138.2s)
 144.3s  ✓ green: check posted on f9add68
        ★ saved as web-main-b@6 in 4.2s: the next job starts here
```

## The complete files

Copy both into a folder on one of your machines, edit `ci.json`, and start with `run ci.run.ts`.

<AccordionGroup>
  <Accordion title="ci.json" icon="braces">
    ```json ci.json theme={"theme":"github-dark"}
    {
      "acme/web": {
        "snapshot": "web-main",
        "workdir": "/home/boxd/work/web",
        "on": { "push": ["main"], "pull_request": true },
        "setup": [
          "git diff --quiet \"$BASE\" HEAD -- yarn.lock || yarn install --frozen-lockfile"
        ],
        "checks": [
          "yarn lint --cache",
          "yarn typecheck --incremental"
        ],
        "fanout": { "command": "yarn test --shard={i}/{n}", "n": 8 },
        "promote": "main"
      }
    }
    ```
  </Accordion>

  <Accordion title="ci.run.ts" icon="file-code">
    ```typescript ci.run.ts theme={"theme":"github-dark"}
    // ci.run.ts — CI that runs on boxd, configured outside GitHub.
    //
    //   run ci.run.ts          (on any of your machines; the platform holds the
    //                           GitHub trigger and wakes it on an event)
    //
    // GitHub only sends the event and shows the result. For every push or pull
    // request on a configured repo:
    //
    //   1. restore a fresh, isolated machine from the repo's snapshot
    //      (main's last green state: checkout, node_modules, caches, Docker layers)
    //   2. fetch and check out the commit; run the setup steps
    //   3. at the same moment, restore one more machine per test shard from the
    //      same snapshot; each checks out and runs its slice of the tests while
    //      the checks run on the first
    //   4. report a GitHub check run on the commit
    //   5. green on main: save the machine as the next version (two snapshot
    //      names, alternating, so the last one stays restorable while one saves).
    //      Anything else is destroyed.
    //
    // No runners, no pools: every job is a restore from a snapshot, in milliseconds.
    import { boxd, githubApp, object } from "@boxd/run";
    import config from "./ci.json";

    type Repo = {
      snapshot: string;
      workdir: string;
      on: { push?: string[]; pull_request?: boolean };
      setup: string[];
      checks: string[];
      fanout?: { command: string; n: number };
      promote?: string;
    };
    const repos = config as Record<string, Repo>;

    /** Which of the two snapshot names jobs restore from, per repo. Survives restarts. */
    const state = object<{ active: Record<string, string> }>("boxd-ci");
    state.active ??= {};
    // A stored name only counts while it belongs to the configured snapshot, so
    // pointing a repo at a new snapshot in ci.json takes effect.
    const active = (repo: string) => {
      const a = state.active[repo];
      return a === repos[repo].snapshot || a === `${repos[repo].snapshot}-b` ? a : repos[repo].snapshot;
    };
    const other = (repo: string) => (active(repo) === repos[repo].snapshot ? `${repos[repo].snapshot}-b` : repos[repo].snapshot);
    const promotions = new Map<string, Promise<void>>();

    const secs = (t0: number) => ((performance.now() - t0) / 1000).toFixed(1);
    const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

    type Step = { name: string; ok: boolean; s: string; tail?: string };

    async function sh(id: string, dir: string, script: string, env?: Record<string, string>, timeout = 30 * 60_000) {
      return boxd.machines.exec(id, { command: ["bash", "-lc", `cd ${dir} && ${script}`], env, timeout });
    }

    async function step(steps: Step[], name: string, id: string, dir: string, script: string, env?: Record<string, string>, say?: (m: string) => void) {
      const t0 = performance.now();
      const r = await sh(id, dir, script, env);
      const out = `${r.stdout}\n${r.stderr}`.trim().split("\n");
      steps.push({ name, ok: r.success, s: secs(t0), tail: r.success ? undefined : out.slice(-40).join("\n") });
      say?.(`${r.success ? "✓" : "✗"} ${name}  (${secs(t0)}s)`);
      return r.success;
    }

    /** The short form of a step for the live log: `yarn test:code --cache …` → `test:code`,
        `pnpm run lint` → `lint`; anything else is the command itself, cut to 48 characters. */
    const short = (cmd: string) =>
      cmd.match(/(?:yarn|pnpm|npm|bun)(?: run)? ([\w:.-]+)/)?.[1] ?? (cmd.length > 48 ? `${cmd.slice(0, 47)}…` : cmd);

    async function ci(repoName: string, sha: string, ref: string, isMain: boolean, label: string, shards?: number) {
      const repo = repos[repoName];
      const [owner, name] = repoName.split("/");
      const gh = await githubApp.client();
      const t0 = performance.now();
      // The live log, `run logs -f`: one line per event, timed from the push.
      const say = (msg: string) => console.log(`${secs(t0).padStart(6)}s  ${msg}`);
      const check = await gh.rest.checks.create({ owner, repo: name, name: "boxd-ci", head_sha: sha, status: "in_progress", started_at: new Date().toISOString() });
      const steps: Step[] = [];
      const machines: string[] = [];
      let ok = false;
      let keep: string | undefined; // the machine to save as the next snapshot
      let restore = "";
      const source = active(repoName); // the snapshot this job restores from
      const n = repo.fanout ? (shards ?? repo.fanout.n) : 0;
      const boots: number[] = [];
      console.log(`\n▶ ${label} · ${sha.slice(0, 7)} · ${n + 1} machines from ${source}`);
      try {
        const token = await githubApp.getToken();
        const jobName = `ci-${sha.slice(0, 7)}-${Date.now() % 100000}`;

        // One machine, restored from main's last green state, with the commit on
        // top. Isolated: no boxd CLI, no integrations, so the code under test can't
        // reach the account. The App token is only in the fetch's environment.
        // `log` decides whether its checkout and setup show up as their own rows.
        const prepare = async (machineName: string, log: boolean) => {
          const tr = performance.now();
          const m = await boxd.machines.create({ name: machineName, fromSnapshot: source, isolated: true });
          machines.push(m.id);
          // A large or fresh snapshot can answer before the machine is up.
          await boxd.machines.waitUntilReady(m.id);
          const restored = `${secs(tr)}s`;
          // create() answers before the boot is recorded: read it back.
          const boot = m.bootTimeMs ?? (await boxd.machines.get(m.id).catch(() => null))?.bootTimeMs ?? null;
          if (boot != null) boots.push(boot);
          else boots.push(-1);
          if (boots.length === n + 1) {
            const known = boots.filter((b) => b >= 0);
            say(`✓ ${n + 1} machines restored from the snapshot${known.length ? `, each booted in ${Math.min(...known)}–${Math.max(...known)}ms` : ""}`);
          }
          const prep: Step[] = log ? steps : [];
          const base = (await sh(m.id, repo.workdir, "git rev-parse HEAD")).stdout.trim(); // the snapshot's commit
          const fetched = await step(prep, `checkout ${sha.slice(0, 7)}`, m.id, repo.workdir,
            `git -c http.extraheader="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GH_APP_TOKEN" | base64 -w0)" fetch -q origin ${sha} && git checkout -q -f ${sha}`,
            { GH_APP_TOKEN: token });
          let ready = fetched;
          for (const s of repo.setup) if (ready) ready = await step(prep, s, m.id, repo.workdir, s, { BASE: base });
          if (!ready && !log) steps.push(...prep.filter((x) => !x.ok).map((x) => ({ ...x, name: `${machineName}: ${x.name}` })));
          return { m, restored, ready };
        };

        // The main machine runs the checks; every test shard gets its own machine,
        // restored from the same snapshot at the same moment. Nothing waits on
        // anything else.
        // A shard that cannot get its machine (an org VM limit, say) fails the
        // run. Left uncaught, its rejection would take the whole automation down.
        const shards = Array.from({ length: n }, (_, i) => (async () => {
          try {
            const { m, ready } = await prepare(`${jobName}-s${i + 1}`, false);
            if (!ready) return false;
            const cmd = repo.fanout!.command.replaceAll("{i}", String(i + 1)).replaceAll("{n}", String(n));
            return await step(steps, `shard ${i + 1}/${n}`, m.id, repo.workdir, cmd, undefined, say);
          } catch (e) {
            steps.push({ name: `shard ${i + 1}/${n}`, ok: false, s: secs(t0), tail: e instanceof Error ? e.message : String(e) });
            return false;
          }
        })());
        const main = await prepare(jobName, true);
        restore = main.restored;
        let checksOk = main.ready;
        if (checksOk) for (const s of repo.checks) if (!(await step(steps, short(s), main.m.id, repo.workdir, s, undefined, say))) { checksOk = false; break; }
        ok = checksOk && (await Promise.all(shards)).every(Boolean);
        const m = main.m;

        // 5. Green on main: this machine becomes the next snapshot, after the
        //    result is reported (see finally). It is kept out of the cleanup.
        if (ok && isMain && repo.promote) keep = m.id;
      } catch (e) {
        steps.push({ name: "boxd-ci", ok: false, s: secs(t0), tail: e instanceof Error ? e.message : String(e) });
      } finally {
        await Promise.all(machines.filter((id) => id !== keep).map((id) => boxd.machines.delete(id).catch(() => {})));
        const total = secs(t0);
        const failed = steps.find((s) => !s.ok);
        const rows = steps.map((s) => `| ${s.ok ? "✓" : "✗"} | \`${s.name}\` | ${s.s}s |`).join("\n");
        await gh.rest.checks.update({
          owner, repo: name, check_run_id: check.data.id,
          status: "completed", conclusion: ok ? "success" : "failure", completed_at: new Date().toISOString(),
          output: {
            title: ok ? `green in ${total}s on boxd` : `failed: ${failed?.name ?? "?"}`,
            summary: `${label}. Restored from \`${source}\` in ${restore || "?"}; ${repo.fanout ? `${n} test shards, each on its own machine restored from the same snapshot; ` : ""}${total}s in total.\n\n| | step | time |\n|---|---|---|\n${rows}`,
            text: failed?.tail ? `\`\`\`\n${failed.tail.slice(-60_000)}\n\`\`\`` : undefined,
          },
        });
        say(`${ok ? "✓ green" : "✗ failed"}: check posted on ${sha.slice(0, 7)}`);
      }
      // Reported first, saved after: nobody waits on the snapshot.
      if (keep) {
        const machine = keep;
        console.log(`        green on main: saving this machine as the next snapshot…`);
        const prev = promotions.get(repoName) ?? Promise.resolve();
        const next = prev
          .then(() => promote(repoName, machine))
          .catch((e) => console.log(`✗ promote: ${e instanceof Error ? e.message : e}`))
          .finally(() => boxd.machines.delete(machine).catch(() => {}));
        promotions.set(repoName, next);
        await next;
      }
    }

    async function promote(repoName: string, machineId: string) {
      const t0 = performance.now();
      const target = other(repoName);
      const before = await boxd.snapshots.get(target).catch(() => null);
      await boxd.snapshots.create(machineId, target);
      // The copy reads from the live machine: keep it until the snapshot is ready.
      let snap = await boxd.snapshots.get(target);
      while (snap.status !== "ready" || (snap.version ?? 0) <= (before?.version ?? 0)) {
        await sleep(1000);
        snap = await boxd.snapshots.get(target);
      }
      state.active = { ...state.active, [repoName]: target };
      console.log(`        ★ saved as ${target}@${snap.version} in ${secs(t0)}s: the next job starts here`);
    }

    for (const [repoName, repo] of Object.entries(repos)) {
      if (repo.on.push?.length) {
        githubApp.on("push", { repo: repoName }, async (e: any) => {
          const branch = String(e.ref).replace("refs/heads/", "");
          if (!repo.on.push!.includes(branch) || e.deleted) return;
          // "[shards=16]" in the commit message overrides the shard count for that run.
          const asked = /\[shards=(\d+)\]/.exec(e.head_commit?.message ?? "");
          await ci(repoName, e.after, branch, branch === repo.promote, `push to ${branch}`, asked ? Number(asked[1]) : undefined);
        });
      }
      if (repo.on.pull_request) {
        githubApp.on("pull_request", { repo: repoName }, async (e: any) => {
          if (!["opened", "synchronize", "reopened"].includes(e.action)) return;
          await ci(repoName, e.pull_request.head.sha, e.pull_request.head.ref, false, `pull request #${e.number}`);
        });
      }
    }
    console.log(`boxd-ci watching ${Object.keys(repos).join(", ")}`);
    ```
  </Accordion>
</AccordionGroup>

## How fast is it

The CI of 10 open-source projects, three ways: one job on a GitHub-hosted runner, the same shards on GitHub-hosted runners, and the setup above. Every machine on both sides has 2 vCPU.

boxd is **6× faster** than the single job and **3× faster** than the sharded one (medians).

| Project | Tests | GitHub, one job | GitHub, sharded | boxd | vs one job | vs sharded |
| - | - | - | - | - | - | - |
| [Django](https://github.com/django/django) | 19,899 | 12m 54s | 3m 06s | **62s** | 12× | 3.0× |
| [Prettier](https://github.com/prettier/prettier) | 102,413 | 9m 38s | 2m 36s | **53s** | 11× | 3.0× |
| [ripgrep](https://github.com/BurntSushi/ripgrep) | 1,222 | 73s | 2m 09s | **8s** | 9× | 15× |
| [GitHub CLI](https://github.com/cli/cli) | 8,445 | 2m 10s | 2m 02s | **15s** | 9× | 8× |
| [Excalidraw](https://github.com/excalidraw/excalidraw) | 2,456 | 9m 48s | 2m 39s | **88s** | 7× | 1.8× |
| [Vue](https://github.com/vuejs/core) | 3,765 | 2m 32s | 86s | **23s** | 7× | 3.7× |
| [React](https://github.com/facebook/react) | 6,990 | 13m 03s | 3m 33s | **2m 06s** | 6× | 1.7× |
| [Pydantic](https://github.com/pydantic/pydantic) | 13,509 | 2m 35s | 1m 58s | **27s** | 6× | 4.3× |
| [Zod](https://github.com/colinhacks/zod) | 8,320 | 4m 54s | 2m 06s | **69s** | 4× | 1.8× |
| [Svelte](https://github.com/sveltejs/svelte) | 7,879 | 6m 24s | 3m 38s | **2m 33s** | 2.5× | 1.4× |

Each run is the project's lint, format, typecheck and test jobs on a push to `main`. Times are medians, from the push to the result.

### Where the speed-up comes from

* **Sharding** helps the big suites on any runner. Django drops from 12m 54s to 3m 06s on GitHub's own runners. It does nothing for small ones: every GitHub job repeats checkout, toolchain setup and install, so ripgrep gets slower when split.
* **The snapshot** is the rest, the "vs sharded" column. A boxd job starts with dependencies, build output and caches already on disk, so a shard is running tests a few seconds after the push.
* **Machine size** matters little. On 8 vCPU machines with 8 shards, 6 of the 10 projects finished within 16% of their 2 vCPU time. Prettier, React and Excalidraw needed 2 to 4 times the shards on 2 vCPU to come close.

<AccordionGroup>
  <Accordion title="How this was measured">
    - **Projects.** Each at its latest commit on 8 October 2026, copied into a private repository. Excalidraw is at `53973c3`.
    - **Scope.** The project's Linux lint, format, typecheck and unit-test jobs, with one toolchain version. Left out: Windows and macOS jobs, end-to-end suites, jobs that need external services, release and docs jobs.
    - **GitHub, one job.** `ubuntu-latest` in a private repository (2 vCPU, 7 GB), with the ecosystem's standard setup action and dependency cache. One warm-up run to fill the cache, then two measured runs.
    - **GitHub, sharded.** The same runners and caches, as one checks job plus one job per test shard. Two measured runs.
    - **boxd.** The setup on this page with 2 vCPU / 8 GiB machines: checks on one, each test shard on its own, all created from the snapshot. One warm-up run, then three measured runs.
    - **Shards.** 8 per project. Prettier uses 16. React and Excalidraw use 32 on boxd and 16 on GitHub, where the plan used for the test runs 20 jobs at once.
    - **Commands.** The same on all three, apart from the shard flag and on-disk cache flags (`eslint --cache`, `tsc --incremental`) on boxd.
    - **Push to main.** A commit with no code change, the steady state between merges.
    - **Timing.** Wall time from the event to the result, median of the measured runs.
  </Accordion>

  <Accordion title="What a run costs">
    At list prices: $0.006 per runner minute on GitHub, billed per job and rounded up to the minute; $0.059 per vCPU-hour and \$0.018 per GiB-hour on boxd, metered per second. The boxd figure assumes every machine holds all 8 GiB for the whole run, so it is an upper bound.

    | Project | GitHub, one job | GitHub, sharded | boxd |
    | - | - | - | - |
    | Django | 7.8¢ | 10.5¢ | **4.1¢** |
    | Prettier | 6.0¢ | 15.9¢ | **6.5¢** |
    | ripgrep | 1.2¢ | 8.7¢ | **0.6¢** |
    | GitHub CLI | 1.8¢ | 10.5¢ | **1.0¢** |
    | Excalidraw | 6.0¢ | 24.3¢ | **21.0¢** |
    | Vue | 1.8¢ | 6.3¢ | **1.5¢** |
    | React | 8.4¢ | 24.9¢ | **30.2¢** |
    | Pydantic | 1.8¢ | 9.3¢ | **1.8¢** |
    | Zod | 3.0¢ | 11.4¢ | **4.5¢** |
    | Svelte | 4.2¢ | 8.4¢ | **10.0¢** |

    Sharding trades compute for time on either platform. boxd costs less than the sharded GitHub run for 8 of the 10 projects, because no shard pays for setup. React and Excalidraw run 33 machines each, twice the GitHub shard count.
  </Accordion>

  <Accordion title="What the numbers don't show">
    * **Sharding has limits.** One slow test module sets Django's time at any shard count. Svelte gains least: one test file holds most of the suite's time, and its build step has no cache.
    * **Checks run in sequence** on the first machine. For Zod they take as long as the tests, so more shards change nothing.
    * **Go caches test results.** On an unchanged commit, GitHub CLI on boxd mostly replays results cached in the snapshot, so its row measures cache coverage more than test speed.
    * **Retries and skips, the same on all three.** Excalidraw, Vue and Zod run with `--retry=2`. Django skips one timing-sensitive test and React one test file (4 tests). React's single hosted job needs two extra Jest flags to fit in 7 GB.
    * **A snapshot carries state.** Prettier's lint leaves a format cache that its own CLI tests race to delete, so the shards remove it first. Pydantic's plugin tests depend on which files share a shard.
  </Accordion>

  <Accordion title="Where the time goes on boxd (Excalidraw)">
    | Step | Time |
    | - | - |
    | Create 33 machines from the snapshot, all at once | about 10s (4s for 9) |
    | Fetch and check out the commit | about 1s |
    | Install | 0.2s, skipped: the lockfile didn't change |
    | Prettier, ESLint, typecheck, with the caches from the snapshot | 5s, 2s, 5s |
    | 32 test shards, in parallel | 37s at the median, 80s for the slowest |

    On the hosted runner the same job spends 50 to 80 seconds on setup (cache restore and install) and runs the tests in 348 to 537 seconds.
  </Accordion>
</AccordionGroup>

## Good to know

* **The slowest shard sets the time.** Split tests by measured duration, or add shards.
* **More machines, not faster cores.** A boxd vCPU is about as fast as a hosted one. The time comes from the snapshot and from shards running side by side.
* **Machine size comes from the snapshot.** Jobs get the size of the machine you saved. Resize it before saving to change that.
* **Steps are plain commands.** Marketplace actions (`uses:`) don't apply.
* **Warm state can hide problems.** A dependency removed from `package.json` can still be in the snapshot, so a job passes where a clean clone would fail. Run a cold build regularly, for example nightly, to catch that.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.