Skip to main content
Reinforcement learning runs the same environment thousands of times, and every episode should begin from an identical state, so that a difference between two runs comes from the policy rather than from the machine underneath it. Reproducing that state by rebuilding it (reinstall, reload, re-seed every library) is slow and fragile, since one unseeded source of randomness or one leftover file breaks the comparison. boxd takes the reconstruction out of the loop. A fork copies a running machine in milliseconds, memory and disk together, so every rollout starts from literally the same bytes: simulator initialized, weights resident, caches warm. What the seed controls is your policy, and what the fork guarantees is the environment.

Eight rollouts in 1.8 seconds: one forked machine per seed, each starting from the same bytes as the source machine.

What the primitives give you

  • Isolation per rollout. Each fork is a hardware-isolated microVM with its own kernel, disk, and network identity. Nothing leaks between workers.
  • Byte-identical starts. Every fork of the same source machine continues from the same instruction, with the same processes, memory, and files.
  • Real-time branching. A fork copies live memory directly, machine to machine, with no snapshot written to disk first. When rollouts branch on the fly, forking skips the round-trip of saving and loading state, which keeps your GPUs busy instead of waiting on environment resets.
  • Stateful resets. A checkpoint rewinds one machine to a saved moment in place, so episode N+1 starts exactly where episode N did.
  • A durable starting state. A snapshot saves the prepared environment under a name that outlives the machine. Give each environment version its own name and it stays comparable across a whole training run.

Prepare one warm source machine

Set the environment up once, on one machine:
The setup cost is paid exactly once. Every rollout after this point skips it.

Fork per rollout

Each rollout forks the source machine, runs its episode, reports its result, and is deleted. The TypeScript and Python SDKs drive this from your training loop:
Forks are copy-on-write, so dozens of them cost close to no extra storage until they start to diverge, and each one lands in under 200ms with the source machine’s full state.

Verify reproducibility

The check is two forks with the same seed:
Both forks start from identical machine state, so any remaining difference comes from your own code (an unseeded RNG in the policy, wall-clock reads, network calls) rather than from the environment.

Reset between episodes

To reuse one machine across episodes instead of forking per rollout, save a checkpoint at the starting state and rewind to it:
Restore rewinds the machine in place, keeping its name and URL, and brings it back byte-identical to the captured moment. That is what makes the next episode comparable to the last.

A starting state that outlives the machine

A checkpoint belongs to its machine and disappears with it. When the starting state needs to outlive the machine, or a teammate needs the same starting point, save a snapshot:
A snapshot keeps only its latest version, and saving again under the same name replaces it. Save each environment version under its own name and do not save over it, and env-v3 still means the same environment a month later. This is how a result stays reproducible after the source machine is gone.

Scaling out

Rollouts are independent, so they scale sideways. Idle machines hibernate on their own after 4 hours without network traffic, which keeps a pool affordable between batches, and a hibernated worker wakes in about 85ms when the next batch starts. The cap is 50 machines per organization by default, extendable on request for fleet-sized runs. See Resources and limits. When a policy executes untrusted or generated code, create the workers with --isolated so a rollout cannot reach your account or your other machines. An isolated machine keeps outbound internet unless you set an egress allowlist. See Sandboxes.

Good to know

Why fork instead of re-seed. Seed-based reproducibility asks every library in the stack to cooperate, and one forgotten RNG quietly breaks it. A fork moves the problem out of your code: the starting state is identical because it is the same memory, so seeds only need to cover the policy’s own choices. Why the source machine stays warm. A fresh machine would reinstall dependencies and reload weights for every rollout, and each of those steps is both wasted time and a chance for two episodes to start from slightly different state. The fork inherits the finished setup, so per-rollout cost is the fork itself. Fork, checkpoint, or snapshot. Fork when you want many parallel copies of one moment. Checkpoint when one machine should return to its own past. Snapshot when the starting state must outlive the machine. Give each environment version its own snapshot name so a training run keeps starting from the same one.