poimen-memory/knowledge/browser-use.md

browser-use — AI Browser Automation

What It Is

• AI agent that controls a real browser via CDP (Chrome DevTools Protocol).
• Describe task in natural language → agent clicks, types, navigates, extracts data.
• Python library for code-level automation OR CLI for agent integration (Claude Code, Cursor, etc).
• Open source, self-hosted. Optional cloud for stealth/scaling.

Installation

# Python library
uv add browser-use
# or: pip install browser-use

# CLI (via uvx, no install needed)
uvx browser-use

# Install Chromium + dependencies
browser-use install

# Register skill for coding agents
browser-use skill install

CLI Usage

# Run inline Python with pre-imported helpers
browser-use <<'PY'
new_tab("https://example.com")
print(page_info())
PY

# Check local Chrome connection
browser-use --doctor

# macOS: approve remote debugging permission
browser-use mac-approve

# Auth for cloud browsers
browser-use auth login

Python Library — Basic Agent

import asyncio
from browser_use import Agent, ChatBrowserUse

async def main():
    agent = Agent(
        task="Find the price of iPhone 16 on Amazon",
        llm=ChatBrowserUse(model='openai/gpt-5.5'),
    )
    result = await agent.run()
    print(result)

asyncio.run(main())

Structured Output

from pydantic import BaseModel, Field
from browser_use import Agent, ChatBrowserUse

class ProductInfo(BaseModel):
    title: str = Field(..., description='Product name')
    price: float = Field(..., description='Price as number')
    url: str = Field(..., description='Product URL')

agent = Agent(
    task="Find the top 3 results for 'mechanical keyboard' on Amazon",
    llm=ChatBrowserUse(model='bu-2-0-mini-preview'),
    output_model_schema=ProductInfo,
)
result = asyncio.run(agent.run())
if result and result.structured_output:
    print(result.structured_output)

Connect to Existing Browser

from browser_use import Agent, Browser, ChatBrowserUse

# Connect to local Chrome with CDP
browser = Browser(cdp_url='http://localhost:9222')
agent = Agent(browser=browser, llm=ChatBrowserUse(), task="...")

CLI Helper Functions

Pre-imported in browser-use CLI context:

• new_tab(url) — open URL in new tab (first navigation per task)
• goto_url(url) — navigate current tab
• page_info() — get current page info
• current_tab() — get current tab info
• list_tabs() — list all open tabs
• switch_tab(target) — switch to existing tab
• activate_tab(target) — bring tab to foreground
• click_at_xy(x, y) — click at viewport coordinates
• js(code) — execute JavaScript in page
• wait_for_load() — wait for page load after navigation
• ensure_real_tab() — skip internal/stale tabs
• cdp("Domain.method", ...) — raw CDP command
• start_recording(name) / stop_recording() — record session

Remote/Cloud Browsers

# Start remote browser (for parallel tasks or stealth)
browser-use <<'PY'
start_remote_daemon("my-task")
PY

# Use remote browser
BU_NAME=my-task browser-use <<'PY'
new_tab("https://example.com")
print(page_info())
PY

# Stop when done (billed until stopped)
browser-use <<'PY'
stop_remote_daemon("my-task")
PY

Page Interaction Workflow

1. Find elements via accessibility tree: cdp("Accessibility.getFullAXTree")["nodes"]
2. Get coordinates: cdp("DOM.getBoxModel", backendNodeId=n) → compute center
3. Click: click_at_xy(x, y)
4. Verify: js(...) or page_info() to confirm action
5. Fallback to raw HTML via js(...) only when AX tree lacks element

When to Use browser-use vs curl

• Use curl: public pages, APIs, static content, docs
• Use browser-use: login-required pages, JS-rendered content, form filling, clicking, bot-protected sites, interactive workflows

Environment Variables

• BROWSER_USE_API_KEY — API key for ChatBrowserUse or cloud
• BU_NAME — name for remote daemon
• BU_CDP_URL — custom CDP endpoint
• BH_DOMAIN_SKILLS=1 — enable domain-specific skills
• BH_AGENT_WORKSPACE — path to agent workspace with helpers

Recordings

# Enable/disable background recording
browser-use recordings enable
browser-use recordings disable

# View latest recording
browser-use recordings --latest

Common Use Cases

• Fill job applications with resume data
• Extract structured data from websites (followers, products, prices)
• Compare prices across multiple sites
• Automate checkout flows
• Screenshot pages for verification
• QA testing of web applications
• Monitor website changes

Integration with Verification Workflow

# After deploying an API, verify it works via browser
browser-use <<'PY'
new_tab("https://api.riotpiao.com/health")
info = page_info()
print(info)  # Should show {"status": "ok"}
PY

# Verify UI renders correctly after deployment
browser-use <<'PY'
new_tab("https://myapp.riotpiao.com")
wait_for_load()
# Check page title
title = js("document.title")
print(f"Page title: {title}")
PY

Gotchas

• First navigation per task must use new_tab(url), not goto_url(url)
• Don't open duplicate tabs — check list_tabs() first
• Login walls: stop and ask user. Exception: SSO if already signed in
• chrome://inspect/#remote-debugging must be enabled for local Chrome
• CDP target order ≠ Chrome's visible tab-strip order
• Always call wait_for_load() after navigation
• Cloud browsers bill until stopped — always stop_remote_daemon() when done