​
Introduction

This guide will walk you through using the Runloop SDK to control a remote computer inside a Runloop Devbox. The Runloop API provides a computer-ready Devbox, enabling AI agents to interact with the system programmatically.

1

Set Up Your Environment

Set up your authentication key:

export RUNLOOP_API_KEY="your-api-key"
2

Install and Initialize the Runloop SDK

First, install the Runloop SDK if you haven’t already:

pip install runloop_api_client

Then, import and initialize the SDK:

from runloop_api_client import Runloop

client = Runloop(bearer_token="your-api-key")

This client object allows interaction with the Runloop API.

3

Create a Devbox and Start the Computer Tool

Create your Devbox, wait for it to be ready, and retrieve the connection details:

# Create a Devbox with a computer instance
computer = client.devboxes.computers.create()

# Wait for the Devbox to be fully running
client.devboxes.await_running(computer.devbox.id)

# Retrieve the computer connection details
devbox_id = computer.devbox.id
display_url = computer.live_screen_url
4

Interacting with the Computer

The computer-ready Devbox offers a suite of Computer Tools for agent interactions. The available actions include:

  • Keyboard interaction: key, type
  • Mouse interaction: mouse_move, left_click, left_click_drag, right_click, middle_click, double_click
  • Screen interaction: screenshot, cursor_position

You can access these tools using the Runloop client as shown below:

# Keyboard usage
client.devboxes.computers.keyboard_interaction(devbox_id, action=action, text=text)

# Mouse usage
client.devboxes.computers.mouse_interaction(devbox_id, action=action, coordinate={"x": int, "y": int})    

# Take a screenshot or retrieve current mouse coordinates
client.devboxes.computers.screen_interaction(devbox_id, action=action)
5

Using API Tools with the Computer Tool

Once you create tools for your agent, you can integrate them with your preferred LLM. Here’s an example of integrating it with Anthropic’s Claude:

import Anthropic

anthropic_client = Anthropic(api_key="your-anthropic-api-key")

response = anthropic_client.messages.create(
    model="claude-3",
    max_tokens=300,
    messages=messages
    tools=[tool],
)

Different LLM providers have their own specific formats and requirements for defining and passing tools. Make sure to reference your LLM provider’s documentation for the correct implementation details of tool schemas and function calling.

6

Properly Freeing Resources

To ensure efficient resource management, always shut down the Devbox when you’re done:

client.devboxes.shutdown(computer.devbox.id)

​
Additional Resources

Was this page helpful?