Start a new ScenarioRun. - Runloop AI Docs

API Reference

Introduction

Devbox

The Devbox Object
Devbox Lifecycle
Devbox File Tools
Devbox Shell Tools
Devbox Network Tools
Devbox Persistence Tools
Devbox Observability Tools
Devbox Add-ons

Blueprint

The Blueprint Object
Blueprint Lifecycle
Blueprint Observability

Repository

The Repository Object
Repository Lifecycle

Code Scenario

Benchmark

Benchmark Lifecycle
Benchmark Runs
Public Benchmarks

POST

scenarios

start_run

import Runloop from '@runloop/api-client';

const client = new Runloop({
  bearerToken: process.env['RUNLOOP_API_KEY'], // This is the default and can be omitted
});

async function main() {
  const scenarioRunView = await client.scenarios.startRun({ scenario_id: 'scenario_id' });

  console.log(scenarioRunView.id);
}

main();

{
  "id": "<string>",
  "name": "<string>",
  "scenario_id": "<string>",
  "devbox_id": "<string>",
  "benchmark_run_id": "<string>",
  "scoring_contract_result": {
    "score": 123,
    "scoring_function_results": [
      {
        "score": 123,
        "scoring_function_name": "<string>",
        "output": "<string>",
        "state": "unknown"
      }
    ]
  },
  "start_time_ms": 123,
  "duration_ms": 123,
  "state": "running",
  "metadata": {}
}

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

scenario_id

string

required

ID of the Scenario to run.

run_name

string | null

Display name of the run.

benchmark_run_id

string | null

Benchmark to associate the run.

metadata

object | null

User defined metadata to attach to the run for organization.

Response

200 - application/json

A ScenarioRunView represents a single run of a Scenario on a Devbox. When completed, the ScenarioRun will contain the final score and output of the run.

string

required

ID of the ScenarioRun.

scenario_id

string

required

ID of the Scenario that has been run.

devbox_id

string

required

ID of the Devbox on which the Scenario is running.

state

enum<string>

required

The state of the ScenarioRun.

Available options:

running,

scoring,

scored,

completed,

canceled,

timeout,

failed

metadata

object

required

User defined metadata to attach to the scenario run for organization.

name

string | null

Optional name of ScenarioRun.

benchmark_run_id

string | null

ID of the BenchmarkRun that this Scenario is associated with, if any.

scoring_contract_result

object

The scoring result of the ScenarioRun.

start_time_ms

integer | null

The time that the scenario started

duration_ms

integer | null

Duration scenario took to run.

Was this page helpful?

List ScenarioRuns.Get a previously created ScenarioRun.

import Runloop from '@runloop/api-client';

const client = new Runloop({
  bearerToken: process.env['RUNLOOP_API_KEY'], // This is the default and can be omitted
});

async function main() {
  const scenarioRunView = await client.scenarios.startRun({ scenario_id: 'scenario_id' });

  console.log(scenarioRunView.id);
}

main();

{
  "id": "<string>",
  "name": "<string>",
  "scenario_id": "<string>",
  "devbox_id": "<string>",
  "benchmark_run_id": "<string>",
  "scoring_contract_result": {
    "score": 123,
    "scoring_function_results": [
      {
        "score": 123,
        "scoring_function_name": "<string>",
        "output": "<string>",
        "state": "unknown"
      }
    ]
  },
  "start_time_ms": 123,
  "duration_ms": 123,
  "state": "running",
  "metadata": {}
}