> ## Documentation Index > Fetch the complete documentation index at: https://docs.runloop.ai/llms.txt > Use this file to discover all available pages before exploring further. # OpenAI Agents API on Runloop > Runbook for running an OpenAI Agents API session against a self-hosted Runloop devbox: keys, provisioning, executor startup, streaming, and teardown. The [OpenAI Agents API](https://developers.openai.com/api/docs/guides/agents-api/overview) is in public beta. Model IDs and event names may change before general availability. The OpenAI Agents API runs the agent model in OpenAI's cloud and connects it to compute you own. On Runloop, that compute is a devbox: your application creates an Agents API session, boots a devbox, starts the Codex executor inside it pointed at the session's environment, and streams the turn. Files the agent writes land on the devbox filesystem, where your application reads them back. Every step below is a stage of the [complete script](#complete-script) at the end of this page, followed by the operational rules that keep sessions and devboxes from leaking. ## Which provisioning mode you are in Pick one mode per session. Do not attach a provisioning webhook handler to sessions your application already provisions. | Mode | Your application | Sandbox provisioning | Guide | | ----------------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ------------------------------------------------- | | **Application-managed** | Calls both the Agents API and the Runloop API. You run a single `main.py`. | Your code creates and shuts down the devbox directly. No handler is deployed. | This page | | **Webhook-managed** | Calls only the Agents API. | A handler you deploy separately starts or reconnects compute when OpenAI sends a provisioning webhook. | Webhook-managed sessions on Runloop (coming soon) | Application-managed is the right mode when your application already has a place to run: it needs no public endpoint, no webhook signature verification, and no separately deployed service. Choose webhook-managed when sessions are created by something that cannot provision compute itself — for example a hosted product surface that only talks to OpenAI. Deleting an Agents API session does **not** stop the devbox. Whichever mode you choose, teardown of Runloop compute is your application's job. ## What you need * A Runloop API key from the [Runloop dashboard](https://platform.runloop.ai/) * An OpenAI account with access to the Agents API * Two OpenAI API keys with the **same owner, organization, and project**: * `OPENAI_API_KEY` — used by your application to create and drive the session * `OPENAI_EXECUTOR_API_KEY` — a separate, restricted key that is the only OpenAI credential permitted inside the devbox * Python 3.11 or newer and [uv](https://docs.astral.sh/uv/) Never inject `OPENAI_API_KEY` into the devbox. The executor key is scoped for exactly this use and is the only one that should cross the sandbox boundary; a leak from inside the devbox should not expose your application's key. ## Environment variables ```bash theme={null} export RUNLOOP_API_KEY=... export OPENAI_API_KEY=sk-... export OPENAI_EXECUTOR_API_KEY=sk-... ``` The example passes the executor key to the devbox through `environment_variables` at create time. For anything beyond a local run, store it as a Runloop account secret and map it in instead — see [Account Secrets](/docs/devboxes/configuration/account-secrets). ## Run it Copy the [complete script](#complete-script) to `openai-agents-api-runloop.py` and run it with [uv](https://docs.astral.sh/uv/): ```bash theme={null} uv run openai-agents-api-runloop.py ``` `uv run` installs the dependencies declared in the script's inline metadata: the official [`openai`](https://github.com/openai/openai-python) SDK (3.13.0 or newer, which is where `client.beta.agents` lives) and `runloop-api-client`. A successful run prints the session ID, the devbox ID, the streamed agent output, and finally the contents of `plan.md`, then shuts the devbox down and deletes the session. ## How the run works The session declares a self-hosted environment and the workspace directory the agent will treat as its working tree. The returned `session.environment` carries the two values the executor needs later: `remote_url` and `id`. ```python theme={null} from openai import AsyncOpenAI WORKSPACE = "/home/user/workspace" async with AsyncOpenAI(timeout=360) as client: session = await client.beta.agents.sessions.create( agent={"model": "gpt-5.6-sol"}, environment={"type": "self_hosted", "workspace_directory": WORKSPACE}, ) assert session.environment.type == "self_hosted" ``` The assert is not decoration: it narrows the environment union so `remote_url` and `id` are typed, and it fails fast if the session came back OpenAI-hosted. Create the devbox through `runloop.api.devboxes.create()`, retain its handle, then wait for it to reach `running`. Keeping the handle before waiting lets cleanup shut it down if boot times out. The executor key goes in as an environment variable named `CODEX_API_KEY`, and `keep_alive_time_seconds` caps the devbox lifetime if the application crashes. ```python theme={null} from runloop_api_client import AsyncRunloopSDK async with AsyncRunloopSDK() as runloop: created = await runloop.api.devboxes.create( name=f"agents-api-{session.id[-12:]}", environment_variables={"CODEX_API_KEY": executor_key}, launch_parameters={"keep_alive_time_seconds": 600}, ) devbox = runloop.devbox.from_id(created.id) await devbox.await_running() ``` Naming the devbox after the session ID is what makes an orphan traceable back to the session that created it. Keep the convention. The devbox side of the connection is `codex exec-server`, installed into the devbox at runtime. Check the exit code — a failed install otherwise surfaces much later as a session that never reaches a completed turn. ```python theme={null} setup = await devbox.cmd.exec( f"mkdir -p {WORKSPACE} && " "npm install --prefix /home/user/.codex-runtime @openai/codex@alpha" ) if setup.exit_code != 0: raise RuntimeError(f"Executor installation failed: {await setup.stderr()}") ``` The `alpha` tag is intentional: the Agents API beta relies on the `exec-server` command, which ships on that tag. OpenAI's [Cookbook sandbox examples](https://github.com/openai/openai-cookbook/tree/main/examples/agents_api/sandboxes) install the same tag for every self-hosted provider. Installing on every run spends devbox time on every session and depends on npm being reachable from the devbox. For repeated runs, bake `@openai/codex` into a [blueprint](/docs/devboxes/blueprints/overview) and create devboxes from that instead. Anything the agent should read has to exist in the workspace before the turn starts. The example writes a one-line brief. ```python theme={null} await devbox.file.write( file_path=f"{WORKSPACE}/brief.txt", contents="Migrate a synchronous Python service to async without changing its API.\n", ) ``` `exec_async` starts the server and returns immediately; the process stays up for the life of the devbox. `--environment-id` is what binds this devbox to the session created in step 1. Pass `session.environment.remote_url` through unchanged rather than hardcoding an endpoint: it is a per-session connect URL, not a fixed API address. ```python theme={null} import shlex CODEX = "/home/user/.codex-runtime/node_modules/.bin/codex" await devbox.cmd.exec_async( f"cd {WORKSPACE} && exec " + shlex.join([ CODEX, "exec-server", "--remote", session.environment.remote_url, "--environment-id", session.environment.id, ]) ) ``` Failure arrives as an event type, not an exception. Treat the five failure events as terminal, and treat a stream that ends without a completed turn as a failure too. `stream()` subscribes before it submits the input, so no events are missed between the two. ```python theme={null} async with client.beta.agents.sessions.stream( session.id, input="Read brief.txt and write a five-step migration plan to plan.md.", ) as events: async for event in events: if event.type in { "error", "agent.session.environment.failed", "agent.session.failed", "agent.session.turn.failed", "agent.session.turn.cancelled", }: raise RuntimeError(f"Agent failed: {event.type}") if event.type == "agent.session.turn.output_text.delta": print(event.delta, end="", flush=True) if ( event.type == "agent.session.turn.completed" and event.turn.subagent_id is None ): break else: raise RuntimeError("Stream ended without a completed turn") ``` Check `turn.subagent_id` before breaking. If the agent delegates work, subagent turns complete on the same stream; breaking on the first `agent.session.turn.completed` would tear the devbox down mid-run. Only the turn with `subagent_id is None` is yours. The agent's real output is the file it wrote, not the streamed text. Read it back and verify it is non-empty — a completed turn does not guarantee the file exists. ```python theme={null} plan = await devbox.file.read(file_path=f"{WORKSPACE}/plan.md") if not plan.strip(): raise RuntimeError("The agent did not write a migration plan") ``` Two independent resources, two cleanups, both in `finally`, and nested so a failed devbox shutdown still deletes the session. If the create request fails before returning an ID, the application cannot target that devbox for cleanup. The fixed lifetime remains the backstop; use the orphan check below after an interrupted run. ```python theme={null} devbox = None try: ... # steps 2 through 7 finally: try: if devbox is not None: async with asyncio.timeout(30): await devbox.shutdown() finally: async with asyncio.timeout(30): await client.beta.agents.sessions.delete(session.id) ``` ## Agents API concepts on Runloop | Agents API | Runloop | Notes | | -------------------------------- | ----------------------------------------- | ------------------------------------------------------------------------------ | | Self-hosted environment | A devbox | One devbox per session in this example | | `workspace_directory` | A path on the devbox | Must exist before the turn starts; the example creates it in the setup command | | `session.environment.id` | `--environment-id` on `codex exec-server` | The only link between session and devbox | | `session.environment.remote_url` | `--remote` on `codex exec-server` | A per-session connect URL; pass it through unchanged | | Executor | `@openai/codex` in the devbox | Authenticates with `CODEX_API_KEY` | | Session deletion | Nothing | The devbox keeps running, and billing, until you shut it down | ## Timeouts and lifetimes The example layers four limits. Enclosing deadlines take precedence over longer SDK timeouts, so adjust the limits together when extending a run. | Limit | Value in the example | Covers | | --------------------------------- | -------------------- | ---------------------------------------------------------------------------- | | `AsyncOpenAI(timeout=360)` | 6 minutes | SDK request timeout; streaming and deletion have shorter enclosing deadlines | | `asyncio.timeout(300)` | 5 minutes | Devbox creation through reading the result | | `asyncio.timeout(30)` per cleanup | 30 seconds each | Devbox shutdown, session deletion | | `keep_alive_time_seconds=600` | 10 minutes | Devbox lifetime if the application dies before cleanup | For idle-based shutdown, set `launch_parameters.lifecycle.after_idle` with `on_idle="shutdown"` and remove `keep_alive_time_seconds`. Mixing the two policies can be rejected or cause the idle policy to take precedence. This bounded batch example uses a fixed lifetime. See [Start and Stop Devboxes](/docs/devboxes/start-stop). ## Multi-turn sessions The example is single-turn: it breaks out of the stream on the main turn's `agent.session.turn.completed` and immediately tears down. For follow-up turns, keep both the session and the devbox alive between turns and call `client.beta.agents.sessions.stream(session.id, ...)` again — the executor is still running and the workspace still holds the previous turn's files. Extend `keep_alive_time_seconds`, or replace it with `lifecycle.after_idle`, so the devbox does not expire between turns. ## Troubleshooting | Symptom | Likely cause | What to do | | ------------------------------------------- | ------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Executor installation failed` | `npm` missing from the image, or npm registry unreachable from the devbox | Confirm `npm --version` runs in the devbox; bake the executor into a blueprint; check any [network policy](/docs/devboxes/blueprints/network-policies) on the blueprint | | `agent.session.environment.failed` | The executor never attached | Verify `exec-server` is still running (`pgrep -af codex`), that `--environment-id` matches the session, and that `CODEX_API_KEY` reached the devbox | | `Stream ended without a completed turn` | The executor exited or lost its connection mid-turn | Pull the devbox execution logs for the `exec-server` process; see [Execution Logs](/docs/devboxes/execution-logs) | | `The agent did not write a migration plan` | Turn completed but the agent wrote elsewhere | Check `workspace_directory` on the session matches the directory the executor was started in | | 401 from the executor | Wrong key, or keys from different projects | Both OpenAI keys must share owner, organization, and project; only the executor key belongs in the devbox | | Devbox still running after the script exits | Cleanup path was skipped | Shut it down explicitly — see below | ## Verify nothing is left running Session deletion and devbox shutdown are independent. After any interrupted run, check for orphans: ```bash theme={null} rli devbox list --output json rli devbox shutdown ``` Devboxes created by this example are named `agents-api-`, which makes them easy to spot. See the [CLI reference](/docs/tools/rl-cli) and [Devbox Lifecycle](/docs/devboxes/lifecycle). ## Complete script Every snippet above is an excerpt of this file. Save it as `openai-agents-api-runloop.py` and run it with `uv run openai-agents-api-runloop.py`. ```python openai-agents-api-runloop.py expandable theme={null} # /// script # requires-python = ">=3.11" # dependencies = [ # "openai>=3.13.0", # "runloop-api-client>=1.31.0", # ] # /// """Run one OpenAI Agents API turn inside a Runloop devbox, then tear down both resources. Requires RUNLOOP_API_KEY, OPENAI_API_KEY, and OPENAI_EXECUTOR_API_KEY. The two OpenAI keys must share an owner, organization, and project. Only the executor key is passed into the devbox. Run with uv: uv run openai-agents-api-runloop.py """ import asyncio import os import shlex from openai import AsyncOpenAI from runloop_api_client import AsyncRunloopSDK MODEL = "gpt-5.6-sol" WORKSPACE = "/home/user/workspace" CODEX_HOME = "/home/user/.codex-runtime" CODEX = f"{CODEX_HOME}/node_modules/.bin/codex" BRIEF = "Migrate a synchronous Python service to async without changing its API.\n" PROMPT = "Read brief.txt and write a five-step migration plan to plan.md." # Failure arrives as an event type, not an exception. FAILURE_EVENTS = { "error", "agent.session.environment.failed", "agent.session.failed", "agent.session.turn.failed", "agent.session.turn.cancelled", } async def main() -> None: executor_key = os.environ["OPENAI_EXECUTOR_API_KEY"] async with AsyncOpenAI(timeout=360) as client, AsyncRunloopSDK() as runloop: session = await client.beta.agents.sessions.create( agent={"model": MODEL}, environment={"type": "self_hosted", "workspace_directory": WORKSPACE}, ) devbox = None print(f"Session: {session.id}", flush=True) try: assert session.environment.type == "self_hosted" async with asyncio.timeout(300): created = await runloop.api.devboxes.create( name=f"agents-api-{session.id[-12:]}", environment_variables={"CODEX_API_KEY": executor_key}, launch_parameters={"keep_alive_time_seconds": 600}, ) devbox = runloop.devbox.from_id(created.id) print(f"Devbox: {devbox.id}", flush=True) await devbox.await_running() setup = await devbox.cmd.exec( f"mkdir -p {WORKSPACE} && " f"npm install --prefix {CODEX_HOME} @openai/codex@alpha" ) if setup.exit_code != 0: raise RuntimeError(f"Executor installation failed: {await setup.stderr()}") await devbox.file.write(file_path=f"{WORKSPACE}/brief.txt", contents=BRIEF) # Binds this devbox to the session; stays up for the life of the devbox. await devbox.cmd.exec_async( f"cd {WORKSPACE} && exec " + shlex.join( [ CODEX, "exec-server", "--remote", session.environment.remote_url, "--environment-id", session.environment.id, ] ) ) async with client.beta.agents.sessions.stream( session.id, input=PROMPT ) as events: async for event in events: if event.type in FAILURE_EVENTS: raise RuntimeError(f"Agent failed: {event.type}") if event.type == "agent.session.turn.output_text.delta": print(event.delta, end="", flush=True) # Subagent turns complete first; only the main turn ends the run. if ( event.type == "agent.session.turn.completed" and event.turn.subagent_id is None ): break else: raise RuntimeError("Stream ended without a completed turn") # The real output is the file the agent wrote, not the streamed text. plan = await devbox.file.read(file_path=f"{WORKSPACE}/plan.md") if not plan.strip(): raise RuntimeError("The agent did not write a migration plan") print(f"\n\nplan.md:\n{plan}") finally: # Deleting the session does not stop the devbox; both need explicit teardown. try: if devbox is not None: async with asyncio.timeout(30): await devbox.shutdown() finally: async with asyncio.timeout(30): await client.beta.agents.sessions.delete(session.id) if __name__ == "__main__": asyncio.run(main()) ``` ## References * [Devbox Lifecycle](/docs/devboxes/lifecycle) * [Execute Commands](/docs/devboxes/execute-commands) * [Blueprints](/docs/devboxes/blueprints/overview) * [Account Secrets](/docs/devboxes/configuration/account-secrets) * [Runloop Python SDK](https://runloopai.github.io/api-client-python/) * [OpenAI Agents API documentation](https://developers.openai.com/api/docs/guides/agents-api/overview) * [Runloop example in the OpenAI Cookbook](https://github.com/openai/openai-cookbook/tree/main/examples/agents_api/sandboxes/application_managed/runloop)