poimen-memory/knowledge/browser-use.md

# browser-use — AI Browser Automation

## What It Is
- AI agent that controls a real browser via CDP (Chrome DevTools Protocol).
- Describe task in natural language → agent clicks, types, navigates, extracts data.
- Python library for code-level automation OR CLI for agent integration (Claude Code, Cursor, etc).
- Open source, self-hosted. Optional cloud for stealth/scaling.

## Installation
```bash
# Python library
uv add browser-use
# or: pip install browser-use

# CLI (via uvx, no install needed)
uvx browser-use

# Install Chromium + dependencies
browser-use install

# Register skill for coding agents
browser-use skill install
```

## CLI Usage
```bash
# Run inline Python with pre-imported helpers
browser-use <<'PY'
new_tab("https://example.com")
print(page_info())
PY

# Check local Chrome connection
browser-use --doctor

# macOS: approve remote debugging permission
browser-use mac-approve

# Auth for cloud browsers
browser-use auth login
```

## Python Library — Basic Agent
```python
import asyncio
from browser_use import Agent, ChatBrowserUse

async def main():
    agent = Agent(
        task="Find the price of iPhone 16 on Amazon",
        llm=ChatBrowserUse(model='openai/gpt-5.5'),
    )
    result = await agent.run()
    print(result)

asyncio.run(main())
```

## Structured Output
```python
from pydantic import BaseModel, Field
from browser_use import Agent, ChatBrowserUse

class ProductInfo(BaseModel):
    title: str = Field(..., description='Product name')
    price: float = Field(..., description='Price as number')
    url: str = Field(..., description='Product URL')

agent = Agent(
    task="Find the top 3 results for 'mechanical keyboard' on Amazon",
    llm=ChatBrowserUse(model='bu-2-0-mini-preview'),
    output_model_schema=ProductInfo,
)
result = asyncio.run(agent.run())
if result and result.structured_output:
    print(result.structured_output)
```

## Connect to Existing Browser
```python
from browser_use import Agent, Browser, ChatBrowserUse

# Connect to local Chrome with CDP
browser = Browser(cdp_url='http://localhost:9222')
agent = Agent(browser=browser, llm=ChatBrowserUse(), task="...")
```

## CLI Helper Functions
Pre-imported in `browser-use` CLI context:
- `new_tab(url)` — open URL in new tab (first navigation per task)
- `goto_url(url)` — navigate current tab
- `page_info()` — get current page info
- `current_tab()` — get current tab info
- `list_tabs()` — list all open tabs
- `switch_tab(target)` — switch to existing tab
- `activate_tab(target)` — bring tab to foreground
- `click_at_xy(x, y)` — click at viewport coordinates
- `js(code)` — execute JavaScript in page
- `wait_for_load()` — wait for page load after navigation
- `ensure_real_tab()` — skip internal/stale tabs
- `cdp("Domain.method", ...)` — raw CDP command
- `start_recording(name)` / `stop_recording()` — record session

## Remote/Cloud Browsers
```bash
# Start remote browser (for parallel tasks or stealth)
browser-use <<'PY'
start_remote_daemon("my-task")
PY

# Use remote browser
BU_NAME=my-task browser-use <<'PY'
new_tab("https://example.com")
print(page_info())
PY

# Stop when done (billed until stopped)
browser-use <<'PY'
stop_remote_daemon("my-task")
PY
```

## Page Interaction Workflow
1. Find elements via accessibility tree: `cdp("Accessibility.getFullAXTree")["nodes"]`
2. Get coordinates: `cdp("DOM.getBoxModel", backendNodeId=n)` → compute center
3. Click: `click_at_xy(x, y)`
4. Verify: `js(...)` or `page_info()` to confirm action
5. Fallback to raw HTML via `js(...)` only when AX tree lacks element

## When to Use browser-use vs curl
- **Use curl**: public pages, APIs, static content, docs
- **Use browser-use**: login-required pages, JS-rendered content, form filling, clicking, bot-protected sites, interactive workflows

## Environment Variables
- `BROWSER_USE_API_KEY` — API key for ChatBrowserUse or cloud
- `BU_NAME` — name for remote daemon
- `BU_CDP_URL` — custom CDP endpoint
- `BH_DOMAIN_SKILLS=1` — enable domain-specific skills
- `BH_AGENT_WORKSPACE` — path to agent workspace with helpers

## Recordings
```bash
# Enable/disable background recording
browser-use recordings enable
browser-use recordings disable

# View latest recording
browser-use recordings --latest
```

## Common Use Cases
- Fill job applications with resume data
- Extract structured data from websites (followers, products, prices)
- Compare prices across multiple sites
- Automate checkout flows
- Screenshot pages for verification
- QA testing of web applications
- Monitor website changes

## Integration with Verification Workflow
```bash
# After deploying an API, verify it works via browser
browser-use <<'PY'
new_tab("https://api.riotpiao.com/health")
info = page_info()
print(info)  # Should show {"status": "ok"}
PY

# Verify UI renders correctly after deployment
browser-use <<'PY'
new_tab("https://myapp.riotpiao.com")
wait_for_load()
# Check page title
title = js("document.title")
print(f"Page title: {title}")
PY
```

## Gotchas
- First navigation per task must use `new_tab(url)`, not `goto_url(url)`
- Don't open duplicate tabs — check `list_tabs()` first
- Login walls: stop and ask user. Exception: SSO if already signed in
- `chrome://inspect/#remote-debugging` must be enabled for local Chrome
- CDP target order ≠ Chrome's visible tab-strip order
- Always call `wait_for_load()` after navigation
- Cloud browsers bill until stopped — always `stop_remote_daemon()` when done