Get a previously created BenchmarkRun.

API Reference

Introduction

Devbox

The Devbox Object
Devbox Lifecycle
Devbox File Tools
Devbox Shell Tools
Devbox Network Tools
Devbox Persistence Tools
Devbox Observability Tools
Devbox Add-ons

Blueprint

The Blueprint Object
Blueprint Lifecycle
Blueprint Observability

Repository

The Repository Object
Repository Lifecycle

Code Scenario

Code Scenario Lifecycle
Scenarios Runs
Custom Scenario Scorer
Public Scenarios

Benchmark

GET

benchmarks

runs

{id}

import Runloop from '@runloop/api-client';

const client = new Runloop({
  bearerToken: process.env['RUNLOOP_API_KEY'], // This is the default and can be omitted
});

async function main() {
  const benchmarkRunView = await client.benchmarks.runs.retrieve('id');

  console.log(benchmarkRunView.id);
}

main();

{
  "id": "<string>",
  "benchmark_id": "<string>",
  "name": "<string>",
  "start_time_ms": 123,
  "duration_ms": 123,
  "state": "running",
  "score": 123,
  "pending_scenarios": [
    "<string>"
  ],
  "scenario_runs": [
    {
      "scenario_id": "<string>",
      "scenarioRunId": "<string>",
      "scoringResult": {
        "score": 123,
        "scoring_function_results": [
          {
            "score": 123,
            "scoring_function_name": "<string>",
            "output": "<string>",
            "state": "unknown"
          }
        ]
      }
    }
  ],
  "metadata": {}
}

Authorizations

Authorization

string

header

required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Path Parameters

string

required

The BenchmarkRun ID.

Response

200 - application/json

A BenchmarkRunView represents a run of a complete set of Scenarios, organized under a Benchmark.

string

required

The ID of the BenchmarkRun.

benchmark_id

string

required

The ID of the Benchmark.

start_time_ms

integer

required

The time the benchmark run execution started (Unix timestamp milliseconds).

state

enum<string>

required

The state of the BenchmarkRun.

Available options:

running,

completed

pending_scenarios

string[]

required

List of Scenarios that need to be completed before benchmark can be completed.

scenario_runs

object[]

required

List of Scenarios have been completed.

A BenchmarkRunScenariosListView represents a run of a complete set of Scenarios run, organized under a Benchmark.

metadata

object

required

User defined metadata to attach to the benchmark run for organization.

name

string | null

The name of the BenchmarkRun.

duration_ms

integer | null

The duration for the BenchmarkRun to complete.

score

number | null

The final score across the BenchmarkRun, present once completed. Calculated as sum of scenario scores / number of scenario runs.

Was this page helpful?

List BenchmarkRuns.Start a new BenchmarkRun.

import Runloop from '@runloop/api-client';

const client = new Runloop({
  bearerToken: process.env['RUNLOOP_API_KEY'], // This is the default and can be omitted
});

async function main() {
  const benchmarkRunView = await client.benchmarks.runs.retrieve('id');

  console.log(benchmarkRunView.id);
}

main();

{
  "id": "<string>",
  "benchmark_id": "<string>",
  "name": "<string>",
  "start_time_ms": 123,
  "duration_ms": 123,
  "state": "running",
  "score": 123,
  "pending_scenarios": [
    "<string>"
  ],
  "scenario_runs": [
    {
      "scenario_id": "<string>",
      "scenarioRunId": "<string>",
      "scoringResult": {
        "score": 123,
        "scoring_function_results": [
          {
            "score": 123,
            "scoring_function_name": "<string>",
            "output": "<string>",
            "state": "unknown"
          }
        ]
      }
    }
  ],
  "metadata": {}
}