Cloud Agent Series – Part 2 – Author: Bao Tran – AI Engineer

In part 1, we explained why we moved our agents into the cloud and gave every run its own microVM. We also argued that choosing the isolation technology is the easy part, and ended with the questions it leaves open: what happens to the conversation when the machine is gone, what a machine keeps between turns, whether a small task needs a full sandbox, and how to keep all of it cheap in production.

This post is about how we answered them in pi-cloud-agent. In short: every run gets a fresh microVM that is deleted when the run ends, the workspace’s files are parked between turns, and a record kept outside the VM lets a task outlive any machine. We got there by starting from what the product promises, not from the infrastructure.

What you keep and what you share

Before settling on a design, we looked at how other cloud agents design their sandbox: Claude Code on the web, Ramp’s Inspect, Amp’s Orbs, Grok Bot and Cloudflare Computer. Their sandboxes look very different, but the differences come down to two questions: what the machine keeps after a task ends, and what it shares with other tasks.

To make that concrete, we split an agent’s computer into four layers: the base image (OS and common tools), the environment (one project’s dependencies and setup), the workspace (the repo plus the agent’s edits) and the runtime (processes and memory right now). For each layer, a product decides whether to keep it or throw it away, and whether it is private to one task or shared. The answers group into four directions.

Drawn from each product's own docs or engineering write-up, linked below, as of October 2026. The line under each direction is what those products promise their users.

Each direction buys something and pays for it somewhere else.

1. Throw the machine away. Every session starts clean from a prepared snapshot and leaves nothing behind. Tasks can’t interfere with each other and an idle session costs nothing, but every start has to be fast, and anything worth keeping has to live outside the machine.

2. Keep the machine. Each thread gets a machine that sleeps when idle and wakes with everything still on it. That suits a collaborator you come back to for weeks. The cost is drift: the machine slowly stops matching everyone else’s setup, and anything bad that lands on it stays.

3. Share the machine. All agents work on one always-on computer with the same files and logins, so handing off work is free. It also means one agent that reads something malicious exposes everything the account can reach.

4. Keep the files, rent compute. Files live in storage, and a container is attached only when the agent needs a real OS. It is the cheapest way to run agents at very large scale, as long as most of the work doesn’t need a full toolchain.

None of these is right in general. Each follows from what the product promises, and that is the lens we used for our own choice.

Why we chose throwaway machines

We give every run a new microVM, started from a prepared snapshot, and delete it when the run ends. Real credentials never go inside: they stay with a broker outside the VM, and outbound traffic goes through an allowlist. Two facts about our product led here. First, nobody watches the runs. A task starts from a prompt, a pull request, a comment or a schedule, and runs to the end without stopping for approvals, so, as we argued in part 1, the sandbox is the only permission system left. Second, some runs start from pull requests by outside contributors, so we have to assume a run can be hostile.

That rules out the other three directions. A machine that sleeps and wakes (2) keeps whatever a malicious pull request leaves on it. A shared machine (3) lets one poisoned task read every other task’s code and secrets. Keeping files apart and attaching compute on demand (4) saves us little, because most of our tasks need a full build-and-test toolchain, so a container would be attached almost every time.

Tasks share nothing, but the turns inside one task do. When someone follows up on a task, the agent shouldn’t forget the file it wrote one message ago. So between turns we park the workspace’s files for up to a week, and each new message starts a new run on a fresh machine from that copy. We keep files, never machines or processes. Harvey landed on almost the same design for Spectre: a follow-up appends to the run and starts a fresh worker.

Each turn gets a new machine, and the second one crashes. Nothing below the VM boundary is lost: the parked files carry the repo and edits from turn to turn, and the record runs straight through the crash.

Making throwaway machines feel permanent

Throwaway machines have a reputation for being slow to start, forgetful and impossible to inspect once they’re gone. Most of our engineering since has gone into those three problems, and all three fixes rest on one idea: separate compute from storage. The machine is rebuilt for every run, and everything that has to last lives outside it.

Everything above the dashed line is rebuilt for every run. Everything below it outlives the machine, and the VM only reaches it across the line. The small labels point to the sections below.

Start time

Each repository in pi-cloud-agent defines a setup script, environment settings and network rules. We run the setup once and save the result as a snapshot, and every run starts from that snapshot instead of from scratch. A warm pool of pre-booted machines, plus booting as soon as a trigger arrives, cuts most of the remaining wait. The network rules double as the run’s egress allowlist, so the same file that makes a run fast also decides what it can reach.

Steps, not timings. We do the same work as a run from scratch, but most of it moves to the left of the trigger, where nobody is waiting for it.

The lesson we took, partly from Ramp and Stripe, who push this further than we do, is that start time is mostly a scheduling problem, not a virtualization one: move every step you can to before the trigger.

This is also our answer to whether a small task needs a full sandbox. Most of our tasks end in a build or a test run, so we start a full VM every time and put our effort into making that cheap rather than avoiding it.

Memory

Every step streams to our database as it happens, and conversation checkpoints are stored outside the sandbox. If a run crashes, everything up to that point is kept, and the next message resumes from the last checkpoint. Because parked files are the only thing that exists between turns, a task feels stateful while we only pay for compute when the agent is actually working.

Visibility

The same stream takes care of visibility. Every run’s transcript is append-only, so you can watch it live or replay it months later, long after its machine is gone. For an agent that nobody watches while it works, this record is the product as much as the code change is. It’s how a reviewer decides whether to trust the pull request.

Back to the questions

Here is where that leaves the questions from part 1.

The takeaway

If you’re designing an agent platform, we’d suggest deciding the isolation technology last. Start with three questions instead.

1. What does the product promise? Fire-and-forget tasks, an ongoing collaborator, a team of bots, massive scale?

2. For each layer, do you keep it, and do you share it?

3. What are the weaknesses of the direction your answers point to, and how will you fix them?

For us, the promise was autonomous, auditable runs that anyone can fire and walk away from. That led us to throwaway machines, and most of our work since has gone into making them feel fast and permanent.

We read a lot while building this. These are the pieces that changed how we think, not a complete survey.

---

Outside reading

Why we built our background agent Ramp Snapshots, warm pools and warming the sandbox while you type. The most concrete write-up on start time we’ve found.

Building Spectre Harvey Disposable workers around a durable run record. Arrived at the same shape as us from a more collaborative starting point.

The two patterns by which agents connect sandboxes Lang ChainAgent inside the sandbox versus sandbox as a tool, and where your secrets end up in each.

Orbs, explained Amp The strongest case for sleep and wake: a machine per conversation that people can share.

Cloudflare Computer Cloudflare The strongest case for files on demand: isolates for most work, containers only when needed.

The design and implementation of Sprites Fly.io Persistent VMs whose disk lives in object storage, and checkpoints cheap enough to use like git.

Memory snapshots Modal How far snapshotting can push start time when you control the whole stack.

awesome-sandbox GitHub A maintained list of sandbox runtimes and platforms, for when you need the full survey we skipped.

pi-cloud-agent is MIT licensed at github.com/zen8labs/pi-cloud-agent. Providers sit behind a small contract, so we’d welcome contributions for other sandbox platforms. Missed the first post? Start with Part 1: Why build a cloud agent?