Skip to main content
The comparison starter implements the bounded checks in the “Run the SDK comparison” section. It does not certify complete migration equivalence. Axons, production networking, and evaluation acceptance require your chosen application infrastructure.

​
Run the SDK comparison

Use Node.js 22 or later and isolated projects. Set RUNLOOP_API_KEY and FREESTYLE_API_KEY in your shell; never commit them. The script creates billable resources and deletes only the VMs, snapshots, and rules it creates.
The default run compares successful and nonzero command exits, stdout/stderr, and UTF-8 file round trips. --extended adds binary and 1 MiB fixtures, permissions and symlinks, a transfer from Runloop to Freestyle and back, outbound denial and explicit HTTPS allowance, disk persistence across suspend/pause and resume, and snapshot restoration on both services. It separately checks Freestyle process survival across pause and snapshot restore. The JSON records package versions, resource IDs, fixture hash, timestamps, assertions, errors, and cleanup. Exit 0 means all implemented assertions and cleanup passed; 1 means a mismatch or cleanup failure; 2 means a probe error. Exit 2 can also include cleanup failures; inspect cleanup on every nonzero exit. Keep the raw report private and inspect failed cleanup for resources requiring removal. Run from the example directory with the committed lockfile.

​
Recorded live run

On October 11, 2026 (PT), the extended probe passed 17/17 assertions with @runloop/api-client 2.0.0 and freestyle 0.2.17. All VM, snapshot, and firewall/DNS cleanup actions succeeded. Separate injected snapshot-stage and devbox-readiness failures returned exit 2 and cleaned up all registered resources. See the sanitized live-run summary for case results and timestamps. The recorded run covers small synthetic fixtures on default Runloop provisioning and Freestyle freestyle/ubuntu-sm. It is evidence for these specific calls and versions, not a production workload, full disk import, Axon replacement, or recovery-time guarantee.

​
Workload acceptance matrix

The filenames in this matrix are suggested names for additional workload checks; those separate scripts are not supplied by the starter. Use the same input fixture on both providers. Record SDK and CLI versions, base image, effective CPU/memory/disk, timestamps, owner, raw output, and exceptions. Normalize only provider IDs and timestamps; preserve exit status, bytes, modes, event order, and scoring values.

​
Evidence and cutover gate

For every required check, record owner, run date, versions, fixture hash, artifact location, result, and approved exception. Require representative workload success, access-policy enforcement, durable state and event recovery, evaluation agreement, and recovery within agreed objectives. Demonstrate reverse transfer for lossless rollback; otherwise document the accepted data-loss window and test recovery to the frozen source state. For deterministic scores, require exact equality unless a tolerance is declared before execution. For stochastic tasks, fix model and sampling settings, use a declared sample count and acceptance threshold, and do not infer equivalence from one run. Check current method signatures in the Runloop SDK documentation and Freestyle’s VM, lifecycle, file, and snapshot guides before extending the starter.