Skip to main content
Start with the Runloop Quickstart to use the examples below.

​
Introduction

This guide will walk you through using the Runloop SDK to control a remote computer inside a Runloop Devbox. The Runloop API provides a computer-ready Devbox, enabling AI agents to interact with the system programmatically.
1

Setup Your Environment

Follow the instructions in the Runloop Quickstart to set up your environment to use the Runloop SDK.
2

Create a Devbox and Start the Computer Tool

Create your Devbox, wait for it to be ready, and retrieve the connection details:
# Create a Devbox with a computer instance                                       
computer = runloop.api.devboxes.computers.create()

# Wait for the Devbox to be fully running
devbox = runloop.devbox.from_id(computer.devbox.id)
await devbox.await_running()

# Retrieve the computer connection details
devbox_id = computer.devbox.id
display_url = computer.live_screen_url
3

Interacting with the Computer

The computer-ready Devbox offers a suite of Computer Tools for agent interactions. The available actions include:
  • Keyboard interaction: key, type
  • Mouse interaction: mouse_move, left_click, left_click_drag, right_click, middle_click, double_click
  • Screen interaction: screenshot, cursor_position
You can access these tools using the Runloop client as shown below:
# Keyboard usage
runloop.api.devboxes.computers.keyboard_interaction(
  devbox_id,
  action=action,
  text=text
)

# Mouse usage
runloop.api.devboxes.computers.mouse_interaction(
  devbox_id,
  action=action,
  coordinate={"x": int, "y": int}
)    

# Take a screenshot or retrieve current mouse coordinates
runloop.api.devboxes.computers.screen_interaction(
  devbox_id,
  action=action
)
4

Using API Tools with the Computer Tool

Once you create tools for your agent, you can integrate them with your preferred LLM. Here’s an example of integrating it with Anthropic’s Claude:
import Anthropic

anthropic_client = Anthropic(
  api_key="your-anthropic-api-key"
)

response = anthropic_client.messages.create(
  model="claude-3",
  max_tokens=300,
  messages=messages
  tools=[tool],
)
Different LLM providers have their own specific formats and requirements for defining and passing tools. Make sure to reference your LLM provider’s documentation for the correct implementation details of tool schemas and function calling.
5

Properly Freeing Resources

To ensure efficient resource management, always shut down the Devbox when you’re done:
await runloop.api.devboxes.shutdown(computer.devbox.id)

​
Additional Resources