> ## Documentation Index > Fetch the complete documentation index at: https://docs.runloop.ai/llms.txt > Use this file to discover all available pages before exploring further. # [Beta] Get a previously created BenchmarkJob. > [Beta] Get a BenchmarkJob given ID. ## OpenAPI ````yaml /openapi-specs/stainless-processed-openapi.json get /v1/benchmark_jobs/{id} openapi: 3.1.0 info: title: RunLoop API version: '0.1' description: >- The RunLoop API spec that allows you to host lambda functions and Devboxes to enable scaled long running ai workflows. contact: name: Runloop AI Support url: https://runloop.ai email: support@runloop.ai servers: - url: https://api.runloop.ai description: Runloop API variables: {} security: - bearerAuth: [] tags: - name: Benchmark - name: Blueprint - name: Blueprint-Lifecycle - name: Blueprint-ObservabilityTools - name: Devbox - name: Devbox-FileTools - name: Devbox-Lifecycle - name: Devbox-NetworkTools - name: Devbox-ObservabilityTools - name: Devbox-PersistenceTools - name: Devbox-ShellTools - name: Scenario - name: ScenarioScorer - name: accounts - name: agents - name: apikeys - name: axons - name: executions - name: gateway-configs - name: mcp-configs - name: network-policies - name: objects - name: restricted_keys - name: secrets - name: streaming paths: /v1/benchmark_jobs/{id}: get: tags: - Benchmark summary: '[Beta] Get a previously created BenchmarkJob.' description: '[Beta] Get a BenchmarkJob given ID.' operationId: getBenchmarkJob parameters: - name: id in: path description: The BenchmarkJob ID. required: true schema: type: string responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/BenchmarkJobView' x-codeSamples: - lang: JavaScript source: |- import Runloop from '@runloop/api-client'; const client = new Runloop({ bearerToken: process.env['RUNLOOP_API_KEY'], // This is the default and can be omitted }); const benchmarkJobView = await client.benchmarkJobs.retrieve('id'); console.log(benchmarkJobView.id); - lang: Python source: |- import os from runloop_api_client import Runloop client = Runloop( bearer_token=os.environ.get("RUNLOOP_API_KEY"), # This is the default and can be omitted ) benchmark_job_view = client.benchmark_jobs.retrieve( "id", ) print(benchmark_job_view.id) components: schemas: BenchmarkJobView: type: object description: >- A BenchmarkJobView represents a benchmark job that runs a set of scenarios entirely on runloop. properties: id: type: string description: The ID of the BenchmarkJob. name: type: string description: The unique name of the BenchmarkJob. state: $ref: '#/components/schemas/BenchmarkJobState' description: The current state of the benchmark job. job_source: description: >- The source configuration that was used to create this job. Either Harbor YAML or benchmark definition reference. anyOf: - $ref: '#/components/schemas/JobSource' - type: 'null' job_spec: description: >- The resolved job specification. Contains scenarios, agents, and orchestrator config. anyOf: - $ref: '#/components/schemas/JobSpec' - type: 'null' failure_reason: description: Failure reason if job failed. type: - string - 'null' benchmark_outcomes: items: $ref: '#/components/schemas/BenchmarkOutcomeView' description: >- Detailed outcome data for each benchmark run created by this job. Includes per-agent results and scenario-level details. type: - array - 'null' in_progress_runs: items: $ref: '#/components/schemas/InProgressRunView' description: >- Benchmark runs currently in progress for this job. Shows runs that have not yet completed. type: - array - 'null' create_time_ms: type: integer format: int64 description: Timestamp when job was created (Unix milliseconds). required: - id - name - state - create_time_ms BenchmarkJobState: type: string enum: - initializing - queued - running - completed - failed - cancelled - timeout JobSource: oneOf: - $ref: '#/components/schemas/HarborJobSource' - $ref: '#/components/schemas/BenchmarkDefJobSource' - $ref: '#/components/schemas/ScenariosJobSource' discriminator: propertyName: type mapping: harbor: $ref: '#/components/schemas/HarborJobSource' benchmark: $ref: '#/components/schemas/BenchmarkDefJobSource' scenarios: $ref: '#/components/schemas/ScenariosJobSource' JobSpec: type: object description: Job specification describing scenarios and execution configuration properties: scenario_ids: type: array items: type: string description: List of scenario IDs to execute orchestrator_config: description: Orchestrator configuration anyOf: - $ref: '#/components/schemas/JobOrchestratorConfig' - type: 'null' agent_configs: type: array items: $ref: '#/components/schemas/JobAgentConfig' description: Agent configurations for this job required: - scenario_ids - agent_configs BenchmarkOutcomeView: type: object description: >- Outcome data for a single benchmark run within a benchmark job, representing results for one agent configuration. properties: benchmark_run_id: type: string description: The ID of the benchmark run. agent_name: type: string description: The name of the agent configuration used. model_name: description: The model name used by the agent. type: - string - 'null' n_completed: type: integer format: int32 description: Number of scenarios that completed successfully. n_failed: type: integer format: int32 description: Number of scenarios that failed. n_timeout: type: integer format: int32 description: Number of scenarios that timed out. average_score: format: float description: Average score across all completed scenarios (0.0 to 1.0). type: - number - 'null' duration_ms: format: int64 description: Total duration of the benchmark run in milliseconds. type: - integer - 'null' scenario_outcomes: type: array items: $ref: '#/components/schemas/ScenarioOutcomeView' description: Detailed outcomes for each scenario in this benchmark run. required: - benchmark_run_id - agent_name - n_completed - n_failed - n_timeout - scenario_outcomes InProgressRunView: type: object description: >- A lightweight view of a benchmark run currently in progress, showing basic execution details without full outcome data. properties: benchmark_run_id: type: string description: The ID of the benchmark run. agent_config: description: >- Agent configuration used for this run. Specifies whether the run was driven by an external API agent or a job-defined agent. anyOf: - $ref: '#/components/schemas/RunAgentConfig' - type: 'null' state: $ref: '#/components/schemas/BenchmarkRunState' description: The current state of the run. start_time_ms: type: integer format: int64 description: Start time (Unix milliseconds). duration_ms: format: int64 description: Duration so far in milliseconds. type: - integer - 'null' required: - benchmark_run_id - state - start_time_ms HarborJobSource: type: object description: Harbor job source with inline YAML configuration properties: inline_yaml: type: string description: The Harbor job configuration as inline YAML content type: type: string enum: - harbor default: harbor required: - inline_yaml - type BenchmarkDefJobSource: type: object description: Benchmark definition job source properties: benchmark_id: type: string description: The ID of the benchmark definition benchmark_name: description: Optional user-provided name for the benchmark definition type: - string - 'null' type: type: string enum: - benchmark default: benchmark required: - benchmark_id - type ScenariosJobSource: type: object description: Scenarios job source with a list of scenario definition IDs properties: scenario_ids: type: array items: type: string description: List of scenario definition IDs to execute type: type: string enum: - scenarios default: scenarios required: - scenario_ids - type JobOrchestratorConfig: type: object description: Orchestrator configuration for benchmark job execution properties: n_concurrent_trials: format: int32 description: >- Number of concurrent trials to run (default: 1). Controls parallelism for scenario execution. Default is 1. type: - integer - 'null' n_attempts: format: int32 description: >- Number of retry attempts on failure (default: 0). This is the retry policy for failed scenarios. Default is 0. type: - integer - 'null' timeout_multiplier: format: float description: >- Timeout multiplier for retries (default: 1.0). Each retry will multiply the timeout by this factor. type: - number - 'null' quiet: description: 'Suppress verbose output (default: false)' type: - boolean - 'null' JobAgentConfig: type: object description: Configuration for an agent in a benchmark job properties: agent_id: description: ID of the agent to use (optional if agent exists by name) type: - string - 'null' name: type: string description: Name of the agent model_name: description: Model name override for this agent type: - string - 'null' timeout_seconds: format: float description: Timeout in seconds for this agent type: - number - 'null' kwargs: additionalProperties: type: string description: Additional kwargs for agent configuration type: - object - 'null' agent_environment: description: Environment configuration to use for this agent anyOf: - $ref: '#/components/schemas/JobAgentEnvironment' - type: 'null' type: type: string enum: - job_agent default: job_agent required: - name - type ScenarioOutcomeView: type: object description: >- Outcome data for a single scenario execution, including its final state and scoring results. properties: scenario_run_id: description: >- The ID of the scenario run. May be absent if the scenario failed during setup before a run was created. type: - string - 'null' scenario_definition_id: type: string description: The ID of the scenario definition that was executed. scenario_name: type: string description: The name of the scenario. state: $ref: '#/components/schemas/ScenarioState' description: The final state of the scenario execution. score: format: float description: >- The score achieved for this scenario (0.0 to 1.0). Only present if state is COMPLETED. type: - number - 'null' duration_ms: format: int64 description: Duration of the scenario execution in milliseconds. type: - integer - 'null' failure_reason: description: >- Failure information if the scenario failed or timed out. Contains exception type and message. anyOf: - $ref: '#/components/schemas/FailureReason' - type: 'null' required: - scenario_definition_id - scenario_name - state RunAgentConfig: oneOf: - $ref: '#/components/schemas/ExternalApiAgentConfig' - $ref: '#/components/schemas/JobAgentConfig' discriminator: propertyName: type mapping: external_api: $ref: '#/components/schemas/ExternalApiAgentConfig' job_agent: $ref: '#/components/schemas/JobAgentConfig' BenchmarkRunState: type: string enum: - running - canceled - completed - failed JobAgentEnvironment: type: object description: Environment configuration for an agent in a benchmark job properties: environment_variables: additionalProperties: type: string description: Environment variables to set when launching the agent. type: - object - 'null' secrets: additionalProperties: type: string description: >- Secrets to inject as environment variables when launching the agent. Map of environment variable names to secret IDs. type: - object - 'null' ScenarioState: type: string enum: - COMPLETED - FAILED - TIMEOUT - CANCELED FailureReason: type: object description: Information about why a scenario execution failed properties: exception_type: type: string description: >- The exception class name (e.g., 'TimeoutException', 'AgentTimeoutError') exception_message: type: string description: The exception message providing context required: - exception_type - exception_message ExternalApiAgentConfig: type: object description: Configuration for externally-driven benchmark runs via API properties: info: description: Placeholder for future external agent metadata type: - string - 'null' type: type: string enum: - external_api default: external_api required: - type securitySchemes: bearerAuth: scheme: bearer type: http ````