> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Runloop to Freestyle validation plan

> Executable SDK checks and workload acceptance tests for a Freestyle transition

<Warning>
  The [comparison starter](https://github.com/runloopai/runloop-examples/blob/a7c53b40554ed5e47ae88e313db08a81fecf19e3/transitions/runloop-to-freestyle/compare_core_public.mjs) implements the bounded checks in the "Run the SDK comparison" section. It does not certify complete migration equivalence. Axons, production networking, and evaluation acceptance require your chosen application infrastructure.
</Warning>

## Run the SDK comparison

Use Node.js 22 or later and isolated projects. Set `RUNLOOP_API_KEY` and `FREESTYLE_API_KEY` in your shell; never commit them. The script creates billable resources and deletes only the VMs, snapshots, and rules it creates.

```sh theme={null}
git clone https://github.com/runloopai/runloop-examples.git
cd runloop-examples
git fetch origin a7c53b40554ed5e47ae88e313db08a81fecf19e3
git checkout a7c53b40554ed5e47ae88e313db08a81fecf19e3
cd transitions/runloop-to-freestyle
npm ci
npm run probe -- --extended --output transition-results_private.json
```

The default run compares successful and nonzero command exits, stdout/stderr, and UTF-8 file round trips. `--extended` adds binary and 1 MiB fixtures, permissions and symlinks, a transfer from Runloop to Freestyle and back, outbound denial and explicit HTTPS allowance, disk persistence across suspend/pause and resume, and snapshot restoration on both services. It separately checks Freestyle process survival across pause and snapshot restore.

The JSON records package versions, resource IDs, fixture hash, timestamps, assertions, errors, and cleanup. Exit `0` means all implemented assertions and cleanup passed; `1` means a mismatch or cleanup failure; `2` means a probe error. Exit 2 can also include cleanup failures; inspect `cleanup` on every nonzero exit. Keep the raw report private and inspect failed cleanup for resources requiring removal. Run from the example directory with the committed lockfile.

## Recorded live run

On October 11, 2026 (PT), the extended probe passed **17/17 assertions** with `@runloop/api-client` **2.0.0** and `freestyle` **0.2.17**. All VM, snapshot, and firewall/DNS cleanup actions succeeded. Separate injected snapshot-stage and devbox-readiness failures returned exit 2 and cleaned up all registered resources. See the [sanitized live-run summary](https://github.com/runloopai/runloop-examples/blob/a7c53b40554ed5e47ae88e313db08a81fecf19e3/transitions/runloop-to-freestyle/validation-summary.json) for case results and timestamps.

The recorded run covers small synthetic fixtures on default Runloop provisioning and Freestyle `freestyle/ubuntu-sm`. It is evidence for these specific calls and versions, not a production workload, full disk import, Axon replacement, or recovery-time guarantee.

## Workload acceptance matrix

The filenames in this matrix are suggested names for additional workload checks; those separate scripts are not supplied by the starter. Use the same input fixture on both providers. Record SDK and CLI versions, base image, effective CPU/memory/disk, timestamps, owner, raw output, and exceptions. Normalize only provider IDs and timestamps; preserve exit status, bytes, modes, event order, and scoring values.

| Suggested test | Direct comparison | Required assertions |
| - | - | - |
| `compare_commands.mjs` | Runloop `devbox.cmd.exec()` versus Freestyle `vm.exec()` | Success, nonzero exit, stdout/stderr, Unicode, binary and large transfers; check bytes, modes, ownership, and symlinks. Extend beyond the starter's small fixtures. |
| `compare_sessions.mjs` | Runloop named shells versus Freestyle PTY or explicit command context | Directory and environment continuity, concurrent command ordering, cancellation, reconnect, background work, and output framing. Independent command calls are not a persistent shell. |
| `compare_lifecycle.mjs` | Runloop suspend/resume versus Freestyle pause/start and stopped boot | Disk persistence on both; process restart on Runloop and process survival on Freestyle. Check idle policy, deletion deadlines, ephemeral behavior, and unexpected termination. |
| `compare_network.mjs` | Runloop egress policy and tunnels versus Freestyle firewall, TLS routes, and forward auth | Allowed and blocked outbound destinations, authorized and anonymous inbound requests, authorizer failure, tenant separation, and wake-up behavior. The starter tests outbound traffic only. |
| `compare_environment.mjs` | Runloop blueprint versus configured Freestyle snapshot | Dependencies, architecture, effective user, paths, permissions, startup, kernel-dependent tools, and absence of runtime secrets from disk and saved memory. |
| `compare_snapshot.mjs` | Runloop disk snapshot versus Freestyle memory-and-disk snapshot | Manifests and database integrity after source termination; process behavior, external data, snapshot retention, capture failure, and recovery within objectives. The starter compares native snapshots, not cross-provider disk import. |
| `compare_axon.mjs` | Runloop event/SQL/Broker APIs versus your journal, database, and workers | Ordered replay, subscriber cursors, concurrent turns, cancellation, retry deduplication, crash recovery, SQL constraints, and atomic failed batches. A missing target component fails acceptance. |
| `compare_evaluations.mjs` | Legacy results versus Harbor or your chosen harness | Fixed passing/failing/timeout cases, per-scorer values, weights, aggregate, error classification, and artifact retention. Confirm a working Freestyle adapter first. |

## Evidence and cutover gate

For every required check, record owner, run date, versions, fixture hash, artifact location, result, and approved exception. Require representative workload success, access-policy enforcement, durable state and event recovery, evaluation agreement, and recovery within agreed objectives. Demonstrate reverse transfer for lossless rollback; otherwise document the accepted data-loss window and test recovery to the frozen source state.

For deterministic scores, require exact equality unless a tolerance is declared before execution. For stochastic tasks, fix model and sampling settings, use a declared sample count and acceptance threshold, and do not infer equivalence from one run.

Check current method signatures in the [Runloop SDK documentation](/docs/tools/sdks) and Freestyle's [VM](https://www.freestyle.sh/docs/vms), [lifecycle](https://www.freestyle.sh/docs/vms/lifecycle), [file](https://www.freestyle.sh/docs/vms/files), and [snapshot](https://www.freestyle.sh/docs/vms/base-snapshots) guides before extending the starter.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.