Update a Benchmark with a set of Scenarios.

Update a Benchmark.

BenchmarkCreateParameters contain the set of paramters to create a Benchmark.

bearerAuth

A BenchmarkView represents a grouped set of Scenarios that together form a Benchmark.

Runloop AI Docs

What is Runloop?

Quickstart: Giving Agents a Development Environment

Overview

Overview of Devboxes

Overview of Devbox Add-ons

Browser add-on

Computer add-on

Make your agent better and more reliable with Runloop's tools for benchmarking.

Overview of Benchmarking on Runloop

Learn how to easily run your agent against popular public benchmarks.

Public Benchmarks

Learn how to create and run custom benchmarks.

Custom Benchmarks

Build Custom Agent Benchmarks with Runloop

Cursor Rules

Explore, experiment with, and test the Runloop API using the Runloop CLI.

Runloop CLI

Manage, monitor, and optimize your AI-powered coding environments with the Runloop Dashboard.

Runloop Dashboard

Use the Runloop SDKs to interact with the Runloop API.

SDKs

LLM Integration Examples

Usage with Common Model Providers and Frameworks

Welcome to the Runloop API Reference Documentation

Introduction

The Devbox Object

The Blueprint Object

The Repository Object

Start and stop a devbox

Start and Stop a Devbox

Execute commands

Execute Commands on a Devbox

Give your AI agent access to modify and interact with files on your devbox.

Read and write files

Read and Write Files on a Devbox

Enable AI agents to work with full projects: access public and private repositories

Mount a code repository

Mount a Code Repository on a Devbox

Create a tunnel to access ports on your Devbox

Open a tunnel to a service

Open a Tunnel to a Service on a Devbox

Configure your Devboxes using predefined sizes

Instance sizes

Configuring Devbox Instance Sizes

Effectively manage and organize large numbers of Devboxes using metadata

Instance metadata

Managing Devbox Metadata

Snapshots

Devbox Snapshots

Blueprints

Devbox Blueprints

Repositories

Reference documentation for the various states a Devbox can be in.

The Devbox Lifecycle

Debug and fix your Blueprint builds in Runloop.

Troubleshooting Blueprint Builds

Securely connect to a remote Runloop Devbox using SSH for debugging

Debugging Agent Output with SSH

Browser Quickstart

Computer Quickstart

List Blueprints.

Starts build of custom defined container Blueprint. The Blueprint will begin in the 'provisioning' step and transition to the 'building' step once it is selected off the build queue., Upon build complete it will transition to 'building_complete' if the build is successful.

Create and build a Blueprint.

Get the details of a previously created Blueprint including the build status.

Get a Blueprint.

Get all logs from the building of a Blueprint.

Get Blueprint build logs.

Preview building a Blueprint with the specified configuration. You can take the resulting Dockerfile and test out your build using any local docker tooling.

Preview Dockerfile definition for a Blueprint.

List all Devboxes while optionally filtering by status.

List Devboxes.

Create a Devbox and begin the boot process. The Devbox will initially launch in the 'provisioning' state while Runloop allocates the necessary infrastructure. It will transition to the 'initializing' state while the booted Devbox runs any Runloop or user defined set up scripts. Finally, the Devbox will transition to the 'running' state when it is ready for use.

Create a Devbox.

Get the latest details and status of a Devbox.

Get Devbox details.

Suspend a running Devbox and create a disk snapshot to enable resuming the Devbox later with the same disk. Note this will not snapshot memory state such as running processes.

Suspend a running Devbox

Resume a suspended Devbox with the disk state captured as suspend time. Note that any previously running processes or daemons will need to be restarted using the Devbox shell tools.

Resume a suspended Devbox

Shutdown a running Devbox. This will permanently stop the Devbox. If you want to save the state of the Devbox, you should take a snapshot before shutting down or should suspend the Devbox instead of shutting down.

Shutdown a running Devbox.

Send a 'Keep Alive' signal to a running Devbox that is configured to shutdown on idle so the idle time resets.

Reset the idle timer of a running Devbox.

Write UTF-8 string contents to a file at path on the Devbox. Note for large files (larger than 100MB), the upload_file endpoint must be used.

Write text file contents to Devbox filesystem.

Read file contents from a file on a Devbox as a UTF-8. Note 'downloadFile' should be used for large files (greater than 100MB). Returns the file contents as a UTF-8 string.

Read text file contents from Devbox filesystem.

Upload file contents of any type (binary, text, etc) to a Devbox. Note this API is suitable for large files (larger than 100MB) and efficiently uploads files via multipart form data.

Upload binary file contents to Devbox filesystem.

Download file contents of any type (binary, text, etc) from a specified path on the Devbox.

Download binary file contents from Devbox filesystem.

Execute a bash command in the Devbox shell, await the command completion and return the output.

Synchronously execute a shell command on a Devbox

Execute the given command in the Devbox shell asynchronously and returns the execution that can be used to track the command's progress.

Asynchronously execute a command via the Devbox shell

Get the latest status of a previously launched asynchronous execuction including stdout/error and the exit code if complete.

Get status of an asynchronous execution on a Devbox.

Create a live tunnel to an available port on the Devbox. Note the port must be made available using Devbox.create.availablePorts. Otherwise, the tunnel will not connect to any running processes on the Devbox.

Create a tunnel to an available port on the Devbox.

Remove a previously opened tunnel on the Devbox.

Remove an open tunnel on the Devbox.

Create an SSH key for a Devbox to enable remote access.

Create an SSH key for a Devbox

List all snapshots of a Devbox while optionally filtering by Devbox ID.

List disk snapshots of a Devbox.

Create a disk snapshot of a devbox with the specified name and metadata to enable launching future Devboxes with the same disk state.

Create a disk snapshot of a running Devbox.

Delete a previously taken disk snapshot of a Devbox.

Delete a disk snapshot of a Devbox.

Get all logs from a running or completed Devbox.

Get Devbox logs.

Tail the logs for the given devbox. This will return past log entries and continue streaming from there. The stream will then continue to stream logs until the connection is closed.

Live Tail Devbox Logs.

List all available repository connections.

List available repository connections.

Create a connection to a Github Repository and trigger an initial inspection of the repo's technical stack and developer environment requirements.

Create a Repository Connection.

Get Repository Connection details including latest inspection status and generated repository insights.

Get Repository Connection details.

Permanently Delete a Repository Connection including any automatically generated inspection insights.

Delete a Repository Connection and associated objects.

Trigger inspection of the latest version of a repository including repo's technical stack and developer environment requirements.

Trigger inspection of the latest version of a repository connection.

List all analyzed versions of a repository connection including automatically generated insights for each version.

API Reference

Devbox

Blueprint

Repository

Code Scenario

Benchmark

Update a Benchmark.

Authorizations

Path Parameters

Body

Response