Computer Use Examples

Explore real-world examples of Claude's computer control capabilities. Each example includes detailed explanations, code samples, and expected results.

Desktop Navigation and Application Launch

Computer Tool Basic Operations

Learn how to navigate the desktop environment and launch applications using Claude's computer control.

Launch Application

User Request
"Open Firefox and navigate to google.com"
Implementation
{ "messages": [{ "role": "user", "content": "Open Firefox and navigate to google.com" }], "tools": [{ "type": "computer_20241022", "name": "computer", "parameters": { "display_width_px": 1024, "display_height_px": 768 } }] }

Expected Behavior

Claude will perform these actions:

  • Take a screenshot to locate the Firefox icon
  • Move cursor to and click the Firefox icon
  • Wait for browser window to appear
  • Click the address bar
  • Type "google.com"
  • Press Enter to navigate

Window Management

Computer Tool Basic Operations

Demonstrate how to manage application windows using keyboard shortcuts.

Window Controls

User Request
"Minimize the current window, then restore it"
Implementation
// Minimize window await computer.key("alt+Tab") // Ensure correct window is focused await computer.key("alt+space") // Open window menu await computer.key("n") // Select minimize // Wait briefly await new Promise(resolve => setTimeout(resolve, 1000)) // Restore window await computer.key("alt+Tab") // Switch to minimized window

Best Practices

  • Use keyboard shortcuts for reliable window management
  • Always verify window state with screenshots
  • Include appropriate delays between actions
  • Handle different window states gracefully

Web Form Interaction

Computer Tool Web Interaction

Demonstrate how to interact with web forms using keyboard navigation and mouse clicks.

Form Navigation and Input

User Request
"Fill out the contact form with my name 'John Doe' and email 'john@example.com'"
Implementation
{ "messages": [{ "role": "user", "content": "Fill out the contact form with my name 'John Doe' and email 'john@example.com'" }], "tools": [{ "type": "computer_20241022", "name": "computer" }] } // Example of tool use response sequence await computer.screenshot() // Verify form visibility await computer.mouse_move([nameFieldX, nameFieldY]) await computer.left_click() await computer.type("John Doe") await computer.key("Tab") // Move to email field await computer.type("john@example.com")

Best Practices

  • Use Tab key for reliable form field navigation
  • Take screenshots to verify field selection
  • Handle form validation feedback
  • Avoid submitting forms unless explicitly requested

Web Navigation and Content Reading

Computer Tool Web Interaction

Navigate web pages and interact with content using keyboard shortcuts and mouse control.

Page Navigation and Reading

User Request
"Go to the documentation page and find the section about computer use tools"
Implementation
// Navigate using keyboard shortcuts await computer.key("ctrl+l") // Focus address bar await computer.type("https://docs.anthropic.com/claude/docs/computer-use") await computer.key("Return") // Wait for page load and take screenshot await computer.screenshot() // Scroll through content await computer.key("Page_Down") await computer.screenshot() // Use find in page await computer.key("ctrl+f") await computer.type("tools") await computer.key("Return")

Important Notes

  • Use keyboard shortcuts for efficient navigation
  • Take regular screenshots to verify page content
  • Handle loading states appropriately
  • Use search functionality for precise content location

File Operations using Text Editor Tool

Text Editor Tool System Interaction

Demonstrate file creation, viewing, and editing using the text editor tool.

Create and Edit File

User Request
"Create a new file called config.json with some basic settings"
Implementation
{ "messages": [{ "role": "user", "content": "Create a new file called config.json with some basic settings" }], "tools": [{ "type": "text_editor_20241022", "name": "str_replace_editor" }] } // Create new file await text_editor.command("create", { "path": "/repo/config.json", "file_text": `{ "setting1": "value1", "setting2": "value2" }` }) // View file content await text_editor.command("view", { "path": "/repo/config.json" }) // Edit specific content await text_editor.command("str_replace", { "path": "/repo/config.json", "old_str": "value1", "new_str": "updated_value" })

Text Editor Tool Features

  • Create new files with initial content
  • View file contents with line numbers
  • Replace specific text strings
  • Insert text at specific lines
  • Undo recent edits

System Commands using Bash Tool

Bash Tool System Interaction

Execute system commands and manage files using the bash tool.

File System Operations

User Request
"List files in the current directory and create a new folder"
Implementation
{ "messages": [{ "role": "user", "content": "List files in the current directory and create a new folder" }], "tools": [{ "type": "bash_20241022", "name": "bash" }] } // List directory contents await bash.command("ls -la /repo") // Create new directory await bash.command("mkdir -p /repo/new_folder") // Verify directory creation await bash.command("ls -la /repo | grep new_folder")

Bash Tool Capabilities

  • Execute common Linux commands
  • Access Python and system packages
  • Run background processes
  • Maintain state across commands
  • Handle command output efficiently

System Settings Adjustment

Computer Tool System Interaction

Interact with system settings using mouse and keyboard controls.

Adjust Display Settings

User Request
"Open system settings and adjust the screen brightness"
Implementation
// Navigate to system settings await computer.screenshot() // Locate settings icon await computer.mouse_move([settingsX, settingsY]) await computer.left_click() // Navigate to display settings await computer.screenshot() // Verify settings window await computer.key("Tab") // Navigate to search await computer.type("display") await computer.key("Return") // Adjust brightness await computer.screenshot() // Locate brightness slider await computer.mouse_move([sliderX, sliderY]) await computer.left_click_drag([newSliderX, sliderY])

Important Considerations

  • Verify each step with screenshots
  • Use keyboard navigation when possible
  • Handle different system configurations
  • Confirm changes were applied successfully