> ## Documentation Index
> Fetch the complete documentation index at: https://docs.runloop.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Quickstart - Controlling a remote computer in a Runloop Devbox

> Learn how to control a computer programmatically inside a Runloop Devbox using the Runloop SDK

export const ExampleRepoLink = props => {
  return <Info><h3><a href={props.link}>Full example on GitHub</a></h3></Info>;
};

<Tip>Start with the [Runloop Quickstart](/docs/tutorials/quickstart) to use the examples below.</Tip>

## Introduction

This guide will walk you through using the **Runloop SDK** to control a remote computer inside a **Runloop Devbox**. The Runloop API provides a **computer-ready Devbox**, enabling AI agents to interact with the system programmatically.

<Steps>
  <Step title="Setup Your Environment">
    Follow the instructions in the [Runloop Quickstart](/docs/tutorials/quickstart) to set up your environment to use the Runloop SDK.
  </Step>

  <Step title="Create a Devbox and Start the Computer Tool">
    Create your Devbox, wait for it to be ready, and retrieve the connection details:

    <CodeGroup>
      ```python Python theme={null}
      # Create a Devbox with a computer instance                                       
      computer = runloop.api.devboxes.computers.create()

      # Wait for the Devbox to be fully running
      devbox = runloop.devbox.from_id(computer.devbox.id)
      await devbox.await_running()

      # Retrieve the computer connection details
      devbox_id = computer.devbox.id
      display_url = computer.live_screen_url
      ```

      ```typescript TypeScript theme={null}
      // Create a Devbox with a computer instance
      const computer = await runloop.api.devboxes.computers.create();

      // Wait for the Devbox to be fully running
      const devbox = await runloop.devbox.fromId(computer.devbox.id);
      await devbox.awaitRunning();

      // Retrieve the computer connection details
      const devboxId = computer.devbox.id;
      const displayUrl = computer.live_screen_url;
      ```
    </CodeGroup>
  </Step>

  <Step title="Interacting with the Computer">
    The **computer-ready Devbox** offers a suite of **Computer Tools** for agent interactions. The available actions include:

    * **Keyboard interaction**: `key`, `type`
    * **Mouse interaction**: `mouse_move`, `left_click`, `left_click_drag`, `right_click`, `middle_click`, `double_click`
    * **Screen interaction**: `screenshot`, `cursor_position`

    You can access these tools using the Runloop client as shown below:

    <CodeGroup>
      ```python Python theme={null}
      # Keyboard usage
      runloop.api.devboxes.computers.keyboard_interaction(
        devbox_id,
        action=action,
        text=text
      )

      # Mouse usage
      runloop.api.devboxes.computers.mouse_interaction(
        devbox_id,
        action=action,
        coordinate={"x": int, "y": int}
      )    

      # Take a screenshot or retrieve current mouse coordinates
      runloop.api.devboxes.computers.screen_interaction(
        devbox_id,
        action=action
      )
      ```

      ```typescript TypeScript theme={null}
      // Keyboard usage
      await runloop.api.devboxes.computers.keyboardInteraction(
        devboxId, 
        { action: action, 
          text: text }
      );

      // Mouse usage
      await runloop.api.devboxes.computers.mouseInteraction(
        devboxId, 
        { action: action,
          coordinate: { x: number, y: number }}
      );

      // Take a screenshot or retrieve current mouse coordinates
      await runloop.api.devboxes.computers.screenInteraction(
        devboxId, 
        { action: action }
      );
      ```
    </CodeGroup>
  </Step>

  <Step title="Using API Tools with the Computer Tool">
    Once you create tools for your agent, you can integrate them with your preferred LLM. Here's an example of integrating it with **Anthropic's Claude**:

    <CodeGroup>
      ```python Python theme={null}
      import Anthropic

      anthropic_client = Anthropic(
        api_key="your-anthropic-api-key"
      )

      response = anthropic_client.messages.create(
        model="claude-3",
        max_tokens=300,
        messages=messages
        tools=[tool],
      )
      ```

      ```typescript TypeScript theme={null}
      import Anthropic from '@anthropic-ai/sdk';

      const anthropicClient = new Anthropic({
        apiKey: 'your-anthropic-api-key'
      });

      const response = await anthropicClient.messages.create({
        model: 'claude-3',
        maxTokens: 300,
        messages: messages,
        tools: [tool]
      });
      ```
    </CodeGroup>

    <Note>
      Different LLM providers have their own specific formats and requirements for defining and passing tools. Make sure to reference your LLM provider's documentation for the correct implementation details of tool schemas and function calling.
    </Note>
  </Step>

  <Step title="Properly Freeing Resources">
    To ensure efficient resource management, **always shut down the Devbox** when you're done:

    <CodeGroup>
      ```python Python theme={null}
      await runloop.api.devboxes.shutdown(computer.devbox.id)
      ```

      ```typescript TypeScript theme={null}
      await runloop.api.devboxes.shutdown(computer.devbox.id);
      ```
    </CodeGroup>
  </Step>
</Steps>

## Additional Resources

* [Runloop GitHub Repository](https://github.com/runloopai/examples) - Explore more examples.
* [Runloop API Documentation](https://docs.runloop.ai) - Official API reference.