Skip to content

Revise exec interface - #5

Open
aron wants to merge 6 commits into
mainfrom
exec
Open

Revise exec interface#5
aron wants to merge 6 commits into
mainfrom
exec

Conversation

@aron

@aron aron commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Reaching a Workspace from a Worker used to go through a reduced stub. It could run a command and wait for the result, but it could not stream output and it did nothing to protect against shell injection. Callers built commands by pasting values into a string, so a name like x; rm -rf / broke out of its argument, and every example grew its own quoting helper to work around it. The durable object that owned the Workspace had the full surface, so the two sides had drifted apart.

This change gives both sides one interface. getWorkspace is a single front door that returns the same client whether you hand it the durable object itself or the object's stub from a Worker.

// From a Worker, against the durable object stub.
using ws = await getWorkspace(env.MyDO.get(id));

// From inside the durable object that owns the Workspace.
using ws = await getWorkspace(this);

The durable object opts in by extending a mixin. It constructs the Workspace and installs the accessor getWorkspace needs, so there is no hand-written method to maintain.

export class MyDO extends withWorkspace(
  class extends DurableObject<Env> {},
  (self) => ({ storage: self.ctx.storage, backends: [/* ... */] }),
) {}

Commands are now escaped by default. shell.exec takes a tagged template that quotes every interpolated value, so a hostile input stays inside its argument.

const file = "my notes.md";
await ws.shell.exec`cat ${file}`;        // runs: cat 'my notes.md'
await ws.shell.exec`echo ${untrusted}`;  // the value can't break out

The plain form still exists for when you need options, and the sh tag is exported so you can escape a command you pass to it.

await ws.shell.exec("npm test", { cwd: "/workspace", encoding: "utf8" });
await ws.shell.exec(sh`cat ${file}`, { cwd: "/workspace" });

The escaping runs on the caller's side on purpose. When a command crosses the boundary to the durable object it travels as a plain value, and a tagged template's raw text is lost in transit, so escaping there would be too late. Building the finished string in the caller is the only place it can happen safely, and the remote stub rejects a raw tagged-template call so the unescaped path can never run by accident.

The same client can now stream a command's output from a Worker, not just wait for its result. Workers RPC carries byte streams but not arbitrary object streams, so the event stream is framed as newline-delimited JSON on the durable-object side and rebuilt into the familiar handle on the Worker side.

for await (const event of await ws.shell.exec`npm test`) {
  if (event.name === "stdout") process.stdout.write(event.value);
}

// Or wait for the whole run.
const { exitCode, stdout } = await (await ws.shell.exec`npm test`).result();

Streaming and waiting for the result are mutually exclusive on a single handle, matching how the in-process handle already behaves. The stream is only started when you actually read from it, so a caller that only wants the result never pays for it.

flowchart LR
  W["Worker: getWorkspace(stub)"] -->|"exec command"| DO["Durable Object"]
  DO -->|"JSON lines over byte stream"| W
  W -->|"rebuilt into events"| C["for await (event of handle)"]
Loading

To verify locally, run the workspace tests, which cover the escaping, the wire codec round trip including binary and multi-byte output, and both the local and remote client paths:

npm test --workspace @cloudflare/workspace

The artifacts example is updated to the new surface: it extends the mixin, reaches the workspace through getWorkspace, and builds its commands with the sh tag instead of a local quoting helper. The shell interface guide and the package README are updated to describe getWorkspace, the tagged template, and why escaping runs caller-side.

The client's passthrough members for the filesystem, git, assets, and artifacts are typed loosely today because the local and remote surfaces are different concrete types. Unifying them behind one declared interface is a good follow-up now that the shape has settled. The new streaming path has unit and integration coverage but has not been run through the long-lived stub-disposal soak harness, which needs a real container; that is worth running before release.

aron-cf added 5 commits June 24, 2026 12:54
Building a shell command by interpolating values into a string is a
shell-injection risk: a path or branch name can break out of its
argument and run arbitrary commands. Every consumer that built
commands by hand grew its own quoting helper to avoid that, which is
easy to forget and noisy at the call site.

Add an sh tagged template that escapes every interpolated value.
Strings and numbers are quoted, arrays are quoted element-by-element
and joined with spaces, and a raw-marked value is spliced in verbatim
for the rare case that genuinely means shell syntax. The static parts
come from strings.raw, so a backslash written in the template reaches
the shell as written. shellQuote is exported for single-argument
quoting outside a template.

WorkspaceShell.exec rejects a tagged-template call with a clear error.
Escaping has to run on the caller's side, because a
TemplateStringsArray's raw property does not survive structured clone
over RPC; the guard keeps the unescaped path from running silently.
Workers RPC carries byte streams with flow control but not an
arbitrary object stream. To project a streaming exec across the
durable-object boundary, the event stream has to be framed as bytes
on one side and parsed back on the other.

Add encodeExecEvents and decodeExecEvents, which frame a stream of
exec events as newline-delimited JSON and inflate it back. Text
chunks ride as JSON strings; binary chunks are base64-encoded so the
frame stays valid JSON regardless of the bytes. The decoder buffers
partial lines so a chunk boundary can fall anywhere. The codec is a
pure unit with no RPC dependency, tested over a round trip including
multi-byte text, non-utf8 bytes, and split chunk boundaries.
Reaching a Workspace from a Worker went through a reduced stub that
only exposed exec().result(), while the durable object that owned the
Workspace used the full surface. The two diverged, and the worker
side could neither stream exec output nor escape interpolated
commands.

Add getWorkspace, a single front door that returns the same client
whether it is handed the durable object itself or the object's stub
from a Worker. A withWorkspace mixin constructs the Workspace, stashes
it on the instance under a private symbol, and declares the
__getWorkspaceStub accessor on the prototype, the only method shape
Workers RPC dispatches to. getWorkspace reads the symbol stash when
present and falls back to the RPC accessor otherwise, so the durable
object needs no hand-written method.

shell.exec on the client takes both a tagged template, escaped through
sh before the command crosses the wire, and the plain command-plus-
options form. The template form defaults to string output; the plain
form is unchanged.

The handle stub now exposes result(), stream(), and kill(). stream()
frames the event stream as JSONL bytes; the client rebuilds a
host-shaped ExecHandle from it, decoding lazily so a result()-only
caller never starts the byte stream. result() and stream consumption
are mutually exclusive, mirroring the host ExecHandle. The exec span
stays open until the handle is consumed by either path, so its
nesting and exit-code attributes are preserved.
Extend the withWorkspace mixin instead of constructing the Workspace
and hand-writing a getWorkspace() method on the durable object. Reach
the Workspace from the endpoint through getWorkspace(stub), and build
shell commands with the sh tag rather than a local quoting helper. The
local shellQuote helper is gone.
Describe reaching a Workspace through getWorkspace from both a Worker
and the owning durable object, building commands with the sh tagged
template and the plain exec form, and why escaping runs on the
caller's side. Update the README durable-object examples to the
withWorkspace mixin.
@aron aron changed the title Revise exec is Jul 27, 2026
@aron
aron requested a review from CharlieHelps July 27, 2026 11:46
@aron-cf

aron-cf commented Jul 27, 2026

Copy link
Copy Markdown

@CharlieHelps can you review this?

@charliecreates charliecreates Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found three blocking regressions in the new unified Workspace client path.

(this as unknown as WorkspaceLocalHost)[WORKSPACE] = new Workspace(options(self));
}

__getWorkspaceStub(): Promise<import("./stub.js").WorkspaceStub> {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

withWorkspace only installs __getWorkspaceStub, but WorkerBackend still reaches its host through WorkspaceServiceProxy.getWorkspace(), which calls env[binding].get(...).getWorkspace() (proxy.ts lines 154–156). The artifacts example now combines this mixin with WorkerBackend while removing its former getWorkspace() method, so every shell command will invoke a nonexistent RPC method. Keep a compatible getWorkspace() alias or migrate the service proxy and worker-backend path in the same change.

// result()-only caller never trips the "already streaming" guard.
{ highWaterMark: 0 },
);
const handle = stream as ReadableStream<WorkspaceExecEvent<E>> & {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The remote rehydration drops the returned WorkspaceExecHandleStub's disposal capability. The reconstructed value is only a ReadableStream with result and kill, so the documented using handle = await ws.shell.exec(...) path throws for Worker callers, and even a completed result()/stream consumption never disposes the remote handle. That leaks the tracked RPC target; if it is never consumed, its disposer is also the code that cancels the child and closes the exec span. Preserve a [Symbol.dispose] bridge (and dispose it on the appropriate terminal/cancel paths) rather than discarding it here.

async () => {
const spawned =
options.encoding === "utf8"
? await this.#ws.shell.exec(command, {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WorkspaceShellClient accepts id and timeoutMs and forwards them over the Worker boundary, but WorkspaceExecOptions omits both and this call only passes cwd, encoding, and backend to the local shell. Consequently getWorkspace(remote).shell.exec(command, { id, timeoutMs }) silently ignores the requested stable id and timeout, unlike the local path. Add these fields to the stub options and forward them in both branches.

@charliecreates

Copy link
Copy Markdown

I reviewed this and requested changes: #5 (review)

The blockers are the broken withWorkspace/WorkerBackend path, missing remote exec-handle disposal, and remote id/timeoutMs options being dropped.

@charliecreates charliecreates Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dependency-backed tests were unavailable because this checkout has no installed dependencies; the findings below are based on direct inspection of the pinned diff and the current workspace/RPC contracts.

— charlied/pr-review

(this as unknown as WorkspaceLocalHost)[WORKSPACE] = new Workspace(options(self));
}

__getWorkspaceStub(): Promise<import("./stub.js").WorkspaceStub> {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep the WorkerBackend accessor available | 🔴 blocking | § correctness

WorkspaceServiceProxy.getWorkspace() still calls stub.getWorkspace(), and WorkerBackend reaches that proxy for every command, but this mixin only installs __getWorkspaceStub(). Any DO using the documented withWorkspace(..., new WorkerBackend(...)) setup therefore fails when the dynamic shell worker tries to obtain its workspace, so shell.exec cannot run. Expose the accessor expected by WorkspaceServiceProxy or update that proxy and its callers to use the mixin's method.

result(): Promise<unknown>;
kill(signal?: "SIGTERM" | "SIGKILL" | "SIGINT" | "SIGHUP"): Promise<void>;
};
Object.defineProperties(handle, {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preserve disposal for remote exec handles | 🔴 blocking | § correctness

rebuildExecHandle forwards only result and kill; it drops the remote handle's [Symbol.dispose]. The remote WorkspaceExecHandleStub is a tracked capnweb capability, so the Worker-side handle[Symbol.dispose]?.() (including the finally in the artifacts example) becomes a no-op and every remote exec handle remains exported on the long-lived session. Forward the disposer and keep the rebuilt handle's type exposing it so callers can release the remote stub.

return encodeExecEvents(source as ReadableStream<WorkspaceExecEvent<ExecEncoding>>);
}

async kill(signal?: "SIGTERM" | "SIGKILL" | "SIGINT" | "SIGHUP"): Promise<void> {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Settle the remote exec span when killing | 🟠 non-blocking | § correctness

WorkspaceShellStub.exec keeps its workspace.shell.exec span pending on consumer.promise, which is resolved only by result(), stream(), or disposal. The new kill() forwards the signal but never claims the handle or resolves that consumer, so the documented await handle.kill() path leaves the span and tracked handle live unless the caller also consumes or disposes it. Resolve the consumer and mark the handle consumed when kill completes, or otherwise close the span on this path.

const handle = await this.#handle;
const reader = (handle as ReadableStream<WorkspaceExecEvent<E>>).getReader();
try {
while (true) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Preserve backpressure across the remote stream | 🟠 non-blocking | § correctness

The start callback drains handle in a while loop and enqueues every event immediately. controller.enqueue() does not wait for the Worker to read, so a large or chatty command is copied into the DO-side stream queue instead of remaining backpressured to the host process; output-sized memory growth can result. Read from the host in response to downstream pull and cancel the reader when the stream is canceled so the advertised streaming path remains bounded.

? this.#ws.shell
.exec(command, {
async () => {
const spawned =

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forward remote execution timeouts | 🟠 non-blocking | § correctness

The public ShellExecOptions accepts timeoutMs, but the RPC stub forwards only cwd, encoding, and backend here. A Worker call such as ws.shell.exec('sleep 30', { timeoutMs: 100 }) therefore reaches WorkspaceShell without its timeout and can run until the backend default instead of the requested bound. Add timeoutMs to the wire-facing options and pass it through.

//
// `R` is the handle type the underlying surface returns (the host
// `ExecHandle` locally, the handle stub remotely).
export interface WorkspaceShellClient<RUtf8, ROpts, RBytes> {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expose remote exec reattachment | 🟠 non-blocking | § repository-guidance

The current docs/05_shell_interface.md contract exposes shell.get(id, { resume }) and ExecHandle.id for reconnecting, but this new client exposes only exec and rebuildExecHandle does not carry the id. A Worker cannot reattach to an in-flight or retained execution after a reconnect, despite the PR's unified client claim. Add the remote get operation and preserve the handle id.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants