Filesystem API

machines.fs works on a machine’s files from your code the way you work on local ones. Every call runs as the machine’s user and sees the same files its commands see, so a file written here is there for the next exec, and the other way round.

from boxd import Boxd

boxd = Boxd()
fs = boxd.machines.fs

fs.write("myapp", "/app/config.json", '{"debug": true}')   # creates /app if needed
data = fs.read("myapp", "/app/config.json")                # bytes
fs.list("myapp", "/app")                                    # [FsEntry, ...]

import { Boxd } from "@boxd-sh/sdk";

const boxd = new Boxd();
const fs = boxd.machines.fs;

await fs.write("myapp", "/app/config.json", '{"debug": true}');  // creates /app if needed
const data = await fs.read("myapp", "/app/config.json");         // Uint8Array
await fs.list("myapp", "/app");                                   // FsEntry[]

The first argument is the machine’s name or id. A sleeping machine is woken first, so there is no need to start it. Isolated machines work too. For copying files and folders from the command line or the console, see File operations.

Every call

Python	TypeScript	What it does
`read(id, path, offset=, length=)`	`read(id, path, { offset, length })`	A file’s contents, or a byte range of them.
`read_stream(id, path, ...)`	`readStream(id, path, ...)`	The same, chunk by chunk.
`write(id, path, data, mode=)`	`write(id, path, data, { mode })`	Replace a file’s contents, creating it and its parent directories. Returns the file’s entry.
`stat(id, path)`	`stat(id, path)`	One file’s, directory’s or symlink’s entry.
`exists(id, path)`	`exists(id, path)`	Whether the path exists.
`list(id, path)`	`list(id, path)`	A directory’s entries, one level deep.
`iter_list(id, path)`	`listIter(id, path)`	The same, entry by entry.
`mkdir(id, path, mode=)`	`mkdir(id, path, { mode })`	Create a directory and any missing parents.
`remove(id, path, recursive=)`	`remove(id, path, { recursive })`	Remove a file, a symlink, or a directory tree.
`rename(id, old, new, overwrite=)`	`rename(id, old, new, { overwrite })`	Move or rename a file or directory.
`realpath(id, path)`	`realpath(id, path)`	The absolute path, with symlinks, `.` and `..` resolved.

The async Python client (AsyncBoxd) has the same calls, awaited, and its write also takes an async iterable.

Paths

A path is either absolute, or relative to the machine user’s home directory, /home/boxd:

You write	It means
`/etc/app/config.yaml`	The path as written.
`notes.md`	`/home/boxd/notes.md`
`src/main.py`	`/home/boxd/src/main.py`

There is no shell in between, so nothing is expanded: ~ and $HOME are ordinary characters in a name, and "~/notes.md" means a directory called ~ inside the home directory.

Writing files

write replaces the whole file. Text is written as UTF-8. Missing parent directories are created, with mode 0755.

fs.write(id, "run.sh", "#!/bin/sh\necho hi\n", mode=0o755)
fs.write(id, "/app/logo.png", open("logo.png", "rb").read())
entry = fs.write(id, "empty.txt", b"")      # a zero-byte file
entry.size, entry.mode                      # 0, 0o644

await fs.write(id, "run.sh", "#!/bin/sh\necho hi\n", { mode: 0o755 });
await fs.write(id, "/app/logo.png", await readFile("logo.png"));
const entry = await fs.write(id, "empty.txt", "");   // a zero-byte file
entry.size, entry.mode;                              // 0, 0o644

What happens to the file’s metadata:

A new file gets mode 0644, or mode if you pass one, and belongs to the machine’s user.
An existing file keeps its mode and owner. Pass mode to change the mode.
A symlink is written through: the link stays a link, and the file it points to gets the new contents, keeping its own mode.

Readers of the file see either the old contents or the new ones, never a mix: the new contents are written beside the file and then swapped in. A few files can’t be swapped without changing what they are, so they are overwritten in place instead, without that guarantee: a file with several hard links, a file owned by another user, a mount point, and a file in a directory the machine’s user can’t write to. The call only succeeds once every byte has arrived. If the upload is cut off, the file is left as it was, except for a file being overwritten in place, which is then incomplete and the error says so.

Reading files

Only regular files can be read. A symlink is followed to its target.

data = fs.read(id, "/var/log/app.log")                          # bytes
head = fs.read(id, "/var/log/app.log", offset=0, length=4096)   # the first 4 KiB
text = fs.read(id, "notes.md").decode()

const data = await fs.read(id, "/var/log/app.log");                         // Uint8Array
const head = await fs.read(id, "/var/log/app.log", { offset: 0, length: 4096 });
const text = new TextDecoder().decode(await fs.read(id, "notes.md"));

A read that is cut off before the end of the file raises an error. It never returns a shortened file.

Large files

read and write hold the whole file in memory. For files of any size, stream instead: read_stream / readStream hands you chunks as they arrive, and write takes chunks to send as they are produced. There is no size limit either way.

def chunks(path, size=1 << 20):
    with open(path, "rb") as f:
        while piece := f.read(size):
            yield piece

fs.write(id, "/data/dataset.tar", chunks("dataset.tar"))   # any iterable of bytes

with open("backup.tar", "wb") as out:
    for chunk in fs.read_stream(id, "/data/backup.tar"):
        out.write(chunk)

import { createReadStream, createWriteStream } from "node:fs";

await fs.write(id, "/data/dataset.tar", createReadStream("dataset.tar"));  // any (async) iterable of Uint8Array

const out = createWriteStream("backup.tar");
for await (const chunk of fs.readStream(id, "/data/backup.tar")) out.write(chunk);
out.end();

Breaking out of a readStream loop early cancels the transfer. In TypeScript the browser entry (@boxd-sh/sdk/web) collects a streamed write in memory before sending it.

Entries: stat, exists, list

stat, list and write describe files with an FsEntry:

Python	TypeScript
`name`, `path`	`name`, `path`	Base name, and the full path.
`type`	`type`	`"file"`, `"directory"`, `"symlink"` or `"other"` (a FIFO, socket or device). A symlink is `"symlink"`.
`target_type`	`targetType`	What the entry resolves to after following symlinks. `None` / `null` for a symlink that leads nowhere.
`is_file`, `is_dir`, `is_symlink`	`isFile`, `isDir`, `isSymlink`	A symlink to a directory is both `is_symlink` and `is_dir`.
`size`, `mode`, `uid`, `gid`	`size`, `mode`, `uid`, `gid`	Of what the entry resolves to.
`mtime_ns`, `mtime`	`mtime`	Modification time.
`symlink_target`	`symlinkTarget`	What a symlink points to, as stored.

entry = fs.stat(id, "/app/current")
entry.is_symlink, entry.is_dir, entry.symlink_target   # True, True, "releases/42"

fs.exists(id, "/app/missing")                           # False

for entry in fs.list(id, "/app"):
    print(entry.name, entry.type, entry.size)

const entry = await fs.stat(id, "/app/current");
entry.isSymlink, entry.isDir, entry.symlinkTarget;    // true, true, "releases/42"

await fs.exists(id, "/app/missing");                  // false

for (const entry of await fs.list(id, "/app")) {
  console.log(entry.name, entry.type, entry.size);
}

exists follows symlinks, so a symlink that leads nowhere does not exist. It is also false for a path the machine’s user isn’t allowed to look at. It still raises when the machine itself can’t be reached. list returns every entry of the directory, in no particular order, with no cap on how many. Large directories arrive in batches: iter_list / listIter gives you each entry as it arrives instead of waiting for all of them.

Directories, renames and removal

fs.mkdir(id, "/app/cache/a/b")                 # like mkdir -p: parents too, existing is fine
fs.mkdir(id, "/app/secrets", mode=0o700)

fs.rename(id, "/app/next", "/app/current")     # replaces /app/current if it exists
fs.rename(id, "a.txt", "b.txt", overwrite=False)

fs.remove(id, "/app/old.txt")
fs.remove(id, "/app/cache", recursive=True)    # a directory and everything in it

fs.realpath(id, "/app/current/../shared")      # "/app/releases/shared"

await fs.mkdir(id, "/app/cache/a/b");                 // like mkdir -p: parents too, existing is fine
await fs.mkdir(id, "/app/secrets", { mode: 0o700 });

await fs.rename(id, "/app/next", "/app/current");     // replaces /app/current if it exists
await fs.rename(id, "a.txt", "b.txt", { overwrite: false });

await fs.remove(id, "/app/old.txt");
await fs.remove(id, "/app/cache", { recursive: true });  // a directory and everything in it

await fs.realpath(id, "/app/current/../shared");      // "/app/releases/shared"

mkdir gives new directories mode 0755, or mode. A file already at the path is an error.
rename with overwrite off fails if the destination exists, instead of replacing it.
remove on a symlink removes the link, never what it points to, even with recursive. A directory needs recursive. The root directory is refused. A recursive removal never follows symlinks out of the tree. Disks mounted inside it are left alone, and the call then fails with cross_device once everything else is gone.
realpath works like Python’s os.path.realpath: the parts of the path that don’t exist yet are kept as written.

Filenames that aren’t UTF-8

On the machine, a filename is a sequence of bytes. Any name works, including ones with tabs, newlines or bytes that aren’t valid UTF-8.

Python decodes names the way os.fsdecode does: bytes that aren’t UTF-8 are kept as surrogate escapes, so passing entry.path back to any call reaches the same file. Every path argument also accepts bytes.
TypeScript decodes name and path as UTF-8 for display, replacing invalid bytes. nameBytes, pathBytes and symlinkTargetBytes hold the exact bytes, and every path argument also accepts a Uint8Array. realpathBytes is realpath as exact bytes.

for entry in fs.list(id, "/data/import"):
    fs.read(id, entry.path)                  # works whatever bytes the name holds

fs.write(id, b"/data/caf\xe9.txt", b"...")   # a Latin-1 name, as bytes

for (const entry of await fs.list(id, "/data/import")) {
  await fs.read(id, entry.pathBytes);         // works whatever bytes the name holds
}

Errors

A failure inside the machine raises a filesystem error that says what went wrong and names the path. Each carries a reason:

`reason`	Meaning	Python	TypeScript `code`
`not_found`	No such file or directory.	`MachineFileNotFoundError`, a `FileNotFoundError`	`ENOENT`
`permission_denied`	The machine’s user may not do this.	`MachinePermissionError`, a `PermissionError`	`EACCES`
`file_exists`	Something is already at the path.	`MachineFileExistsError`, a `FileExistsError`	`EEXIST`
`is_a_directory`	A directory where a file was needed.	`MachineIsADirectoryError`, an `IsADirectoryError`	`EISDIR`
`not_a_directory`	A file where a directory was needed, here or in a parent.	`MachineNotADirectoryError`, a `NotADirectoryError`	`ENOTDIR`
`directory_not_empty`	The directory has entries.	`MachineFilesystemError`	`ENOTEMPTY`
`not_a_regular_file`	A read of a FIFO, socket or device.	`MachineFilesystemError`	none
`symlink_loop`	Too many levels of symlinks.	`MachineFilesystemError`	`ELOOP`
`name_too_long`	A name or path is too long.	`MachineFilesystemError`	`ENAMETOOLONG`
`invalid_path`	The path can’t be used (for example, it holds a NUL byte).	`MachineFilesystemError`	`EINVAL`
`no_space`	The disk is full.	`MachineFilesystemError`	`ENOSPC`
`read_only`	The filesystem is read-only.	`MachineFilesystemError`	`EROFS`
`busy`	The file is in use, like a mount point or a running program.	`MachineFilesystemError`	`EBUSY`
`cross_device`	The operation can’t cross from one disk to another.	`MachineFilesystemError`	`EXDEV`
`size_mismatch`	Fewer or more bytes arrived than were sent. The file is unchanged.	`MachineFilesystemError`	none
`io_error`	Anything else the disk reported.	`MachineFilesystemError`	`EIO`

In Python, every filesystem error is a MachineFilesystemError, which is both a BoxdError and an OSError, with errno, strerror and filename set the way a local failure sets them. The five common ones are also the matching builtin, so code written for local files handles them unchanged:

try:
    config = fs.read(id, "/app/config.json")
except FileNotFoundError:
    config = b"{}"
except PermissionError as e:
    print(e.filename, e.reason)        # "/app/config.json", "permission_denied"

In TypeScript, every filesystem error is a FilesystemError. Its code is the error name Node’s own fs uses, reason is the exact reason, and path is the path the call was about. The five common ones throw a subclass of it, so you can test the class or the code:

import { FilesystemError, MachineFileNotFoundError } from "@boxd-sh/sdk";

try {
  await fs.read(id, "/app/config.json");
} catch (e) {
  if (e instanceof MachineFileNotFoundError) { /* missing */ }
  else if (e instanceof FilesystemError && e.code === "EACCES") { /* not allowed */ }
  else throw e;
}

Errors about the machine rather than the path are the SDK’s usual ones, for example NotFoundError for a machine that doesn’t exist. In Python the five common filesystem classes also extend the matching status class (MachineFileNotFoundError is a NotFoundError, MachinePermissionError a PermissionDeniedError, the other three a ConflictError), so catch the filesystem class when you need to tell a missing file from a missing machine. In TypeScript FilesystemError extends BoxdError only. See the error sections of the Python and TypeScript references.

Machines started before the filesystem API

A machine that was last started before machines.fs was released can’t answer these calls. They raise MachineAgentOutdatedError in Python and AgentOutdatedError in TypeScript, with reason agent_outdated. Rebooting the machine fixes it, and only needs doing once:

from boxd import MachineAgentOutdatedError

try:
    fs.read(id, "/app/config.json")
except MachineAgentOutdatedError:
    boxd.machines.reboot(id)

import { AgentOutdatedError } from "@boxd-sh/sdk";

try {
  await fs.read(id, "/app/config.json");
} catch (e) {
  if (e instanceof AgentOutdatedError) await boxd.machines.reboot(id);
  else throw e;
}

This error is not a filesystem error (not an OSError in Python, not a FilesystemError in TypeScript): nothing was attempted on the machine’s disk.

Getting started

Guides

Integrations and automations

Use cases

References

Filesystem API

Every call

Paths

Writing files

Reading files

Large files

Entries: stat, exists, list

Directories, renames and removal

Filenames that aren’t UTF-8

Errors

Machines started before the filesystem API

​Every call

​Paths

​Writing files

​Reading files

​Large files

​Entries: stat, exists, list

​Directories, renames and removal

​Filenames that aren’t UTF-8

​Errors

​Machines started before the filesystem API

Every call

Paths

Writing files

Reading files

Large files

Entries: stat, exists, list

Directories, renames and removal

Filenames that aren’t UTF-8

Errors

Machines started before the filesystem API