# Sandboxes for RL rollouts at scale

Run your whole RL loop on one platform, GPUs included.

## RL compute environments

Deterministic resets, parallel rollouts, long-horizon state, and GPUs on-demand.

### Snapshot and restore
Capture the environment's memory and filesystem once and boot every sandbox from that same state.

### Rollouts in parallel
Fork a snapshot into a fleet of isolated sandboxes, each with its own filesystem, processes, and ports.

### Sub-second boots
Sandboxes cold boot in under a second so startup times never bottleneck your evals.

### No timeouts
Environments run as long as the task demands, no timeouts.

### Statefulness & Persistence
Read and write to persistent volumes and create snapshots to restore any sandbox state in the future.

### One Platform
Run evals and retrain your models, all in one place.

## Easily run RL rollouts

### Run Docker-in-Docker
Run any existing Docker image in a sandbox, giving you a fully reproducible environment between your local and the cloud.

### Fan out sandboxes
Spin up thousands of sandboxes, and write outputs to a shared volume.

### Attach GPUs
Add a GPU to any sandbox with just a single line of code.

```python
# Run the policy update on a GPU — same SDK
from beam import function, Volume

@function(
    gpu="H100",
    volumes=[Volume(name="trajectories", mount_path="./data")],
)
def update_policy():
    batch = load_trajectories("./data")
    loss = policy.train_step(batch)
    save_checkpoint("./data/policy-v43.pt")

# Pause a stateful environment and pick it back up later
snapshot_id = env.snapshot_memory()
env2 = Sandbox().create_from_memory_snapshot(snapshot_id)
```

## Frequently asked questions

**How fast can a snapshot restore?**  
A restore is just booting a new sandbox from your saved snapshot, which takes under a second with dependencies included. Every episode starts from the identical filesystem state, and nothing from the previous episode leaks in.

**Can I pause and resume an episode mid-run?**  
Yes. Memory snapshots capture the sandbox's full state — running processes and exposed ports included — and create_from_memory_snapshot picks up exactly where the episode left off. Useful for long-horizon tasks and branching a live trajectory.

**How many rollouts can run at once?**  
Fleets are created programmatically, so parallelism scales with your training run. Each sandbox is isolated with its own filesystem and processes, and billing is per-second — terminated environments cost nothing.

**Can environments have GPUs?**  
Yes. Pass gpu to the Sandbox constructor for environments that render or run models, and keep CPU-only environments on fractional CPUs so large fleets stay cheap.

**Where does the training step run?**  
On Beam, next to the rollouts. Decorate your update function with @function(gpu=...) and mount the volume your environments write to — rollouts, trajectories, and gradient steps stay on one platform.