LumaBrowser Documentation

LumaBrowser is a programmable browser that exposes four local control surfaces on your machine: a REST API on http://localhost:3000, a Model Context Protocol server for AI agents, a W3C WebDriver server on port 9515 for Selenium, and a Chrome DevTools Protocol server on port 9222 for Puppeteer, Playwright, and Playwright MCP. The WebDriver and CDP servers are off by default; turn them on in Settings. Every selector call can fall back to an LLM when the DOM changes. Install with one command and start automating in minutes.

Installation

One-command install (recommended). If you have Node 18+ installed, run:

npx lumabrowser start

The launcher reads the release manifest on lumabyte.com, downloads the right build for your OS from GitHub Releases on first run, caches it under ~/.lumabrowser/, and launches the app. Subsequent runs launch instantly from cache. Expect a 150 to 250 MB download the first time.

All launcher commands:

Want to drop the npx? Install globally:

npm install -g lumabrowser
lumabrowser start

Native installers. If you'd rather have a native installer with a desktop shortcut and Start-menu/Applications entry, grab one for your platform from the downloads page. Available for Windows, macOS (Intel & Apple Silicon), and Linux.

Quick Start

Once running, the REST API is available on http://localhost:3000. Try it:

# Check health
curl http://localhost:3000/api/health

# Create a tab
curl -X POST http://localhost:3000/api/browser/tabs \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Take a screenshot
curl http://localhost:3000/api/browser/tabs/0/screenshot

Pick your automation library

LumaBrowser speaks three automation protocols out of the box. All three share the same LLM selector-fallback pipeline, so the library you pick is a stylistic choice, not a capability tradeoff. Start the driver you want (it's off by default), then connect from any language binding.

# 1. Start the WebDriver server (port 9515) inside LumaBrowser:
curl -X POST http://localhost:3000/api/selenium/start

# 2. Connect from any Selenium binding:
from selenium import webdriver
driver = webdriver.Remote(
  command_executor="http://127.0.0.1:9515",
  options=webdriver.ChromeOptions(),
)
driver.get("https://example.com")

// 1. Start the CDP server (port 9222) inside LumaBrowser:
curl -X POST http://localhost:3000/api/cdp/start

// 2. Connect with Puppeteer (or puppeteer-core):
const puppeteer = require('puppeteer-core');
const browser = await puppeteer.connect({
  browserURL: 'http://127.0.0.1:9222',
});
const page = await browser.newPage();
await page.goto('https://example.com');

// 1. Start the same CDP server (port 9222):
curl -X POST http://localhost:3000/api/cdp/start

// 2. Connect with Playwright via connectOverCDP:
const { chromium } = require('playwright');
const browser = await chromium.connectOverCDP('http://127.0.0.1:9222');
const context = browser.contexts()[0] ?? await browser.newContext();
const page    = await context.newPage();
await page.goto('https://example.com');

See Selenium vs Puppeteer vs Playwright for the full side-by-side, or jump to Selenium Driver / CDP Driver for the per-protocol deep dives.

Configuration

Settings are persisted in SQLite. Key options:

Environment variable override: LUMA_API_PORT=8080 wins over the saved core.apiPort. The REST API needs no key by default; set an API key in Settings before you expose it beyond localhost.

The LumaBrowser REST API ships with the desktop app and runs on http://localhost:3000. When this doc says "the local API" or uses localhost:3000, assume the app is running on the same machine. The "Free APIs" section at the bottom of this page is a separate surface hosted at https://lumabyte.com; those endpoints are not served by the desktop app. See API hosts & what lives where.

Health & Discovery

Health response (actual shape):

{
  "status": "ok",
  "timestamp": "2026-04-19T12:34:56.000Z",
  "port": 3000,
  "mcpEnabled": true,
  "extensions": ["ai-chat", "network-watcher", "timed-tasks",
                 "page-change-detector", "selenium-driver", "cdp-driver"]
}

The extensions array lists the extensions that registered REST routes in this run; extensions that ship on disk but are not loaded will not appear.

Discovery response (actual shape):

{
  "name": "LumaBrowser API",
  "port": 3000,
  "mcpEnabled": true,
  "routes": {
    "core": ["/api/health", "/api/browser",
             "/api/mcp/tools", "/api/mcp/call"],
    "extensions": {
      "ai-chat":              "/api/ai-chat",
      "network-watcher":      "/api/watchers",
      "timed-tasks":          "/api/timed-tasks",
      "page-change-detector": "/api/page-change-detector",
      "personal-hub":         "/api/hub",
      "agent-manager":        "/api/ext/agent-manager",
      "mcp-connector":        "/api/ext/mcp-connector",
      "selenium-driver":      "/api/selenium",
      "cdp-driver":           "/api/cdp"
    }
  }
}

Discovery lists mount points, not every route. To enumerate individual endpoints under each mount (for example /api/browser/tabs/:id/click), refer to the per-section tables on this page. An extension that does not declare its own prefix is mounted at /api/ext/<extension id>. The MCP routes (/api/mcp/tools, /api/mcp/call) appear under routes.core only when the MCP server is wired.

Response Envelope

Most endpoints wrap their payload in one of two envelopes. Legacy endpoints may omit the wrapper entirely; always check the table below before writing a strongly-typed client.

Canonical envelope (newer / core endpoints):

{
  "success": true,
  "timestamp": "2026-04-19T12:34:56.000Z",
  "data": { ... },         // or sometimes an array
  "message": "Optional human-readable context"
}

Payload key by endpoint. Several older extension endpoints use a resource-named key instead of data. Clients that auto-detect the payload key should consult this table to be safe:

ID types

Resource identifiers are not uniformly typed. Strongly-typed clients (C#, Kotlin, Swift, Rust) should model each one explicitly or silent deserialization failures will occur.

Gotcha: Endpoints that take a tab reference expect integer tabId in the JSON body; prefer the integer form over a stringified ID to avoid surprises.

API hosts - what lives where

There are two distinct surfaces. Pointing a client at the wrong host produces a confusing 404.

Full programmatic control over browser tabs, navigate, click, fill forms, extract content, and take screenshots.

Tab Management

Create Tab

When silent: true, the tab is created but not rendered to the user-visible surface. This keeps background automation out of the user's way, but see the silent-tab caveats before using it for screenshot or scroll work.

POST /api/browser/tabs
{
  "url": "https://example.com",
  "silent": false
}

PATCH a Tab - the action discriminator

PATCH /api/browser/tabs/:id requires an action field. A naked {"url": "..."} is rejected with "Action must be one of: navigate, refresh, activate, executeJs". The four supported shapes:

# Navigate an existing tab
PATCH /api/browser/tabs/0
{"action": "navigate", "url": "https://example.com"}

# Refresh
PATCH /api/browser/tabs/0
{"action": "refresh"}

# Foreground (activate) the tab, "focus" is accepted as an alias
PATCH /api/browser/tabs/0
{"action": "activate"}

# Execute JavaScript in the tab
PATCH /api/browser/tabs/0
{"action": "executeJs", "script": "(function(){ return document.title; })()"}

Activate a Tab (make it the visible surface)

Activation foregrounds a tab so it becomes the rendered display surface. This is required before any operation that depends on an actual painted viewport; see silent-tab caveats for the full list. Two equivalent shapes are available:

# Preferred: dedicated endpoint
POST /api/browser/tabs/0/activate

# Or: PATCH with action discriminator (accepts "activate" or "focus" alias)
PATCH /api/browser/tabs/0
{"action": "activate"}

Silent tabs cannot be activated. A tab created with silent: true has no render surface to promote. Calling activate on one returns 400 with {"error": "Silent tabs cannot be activated"}. If you need to foreground work, create the tab with silent: false in the first place.

Reliable screenshot / scroll workflows in multi-tab scripts. Activate the target tab immediately before GET /tabs/:id/screenshot or POST /tabs/:id/scroll. Without this step, operations that require the painted surface can fail on any tab that isn't currently foregrounded.

# Activate, then screenshot - the reliable pattern
curl -X POST http://localhost:3000/api/browser/tabs/2/activate
curl http://localhost:3000/api/browser/tabs/2/screenshot

executeJs: no top-level return. Scripts are evaluated at page top level, where return is a syntax error. A bare "script": "return 2+40" fails with "Script failed to execute". Wrap any return value in an IIFE:

# Wrong
{"action": "executeJs", "script": "return 2+40"}

# Right
{"action": "executeJs", "script": "(function(){ return 2+40; })()"}

# Also fine - last expression is returned
{"action": "executeJs", "script": "2+40"}

Page Inspection

User Interaction

All selector-based endpoints accept an optional llmFallback parameter, a natural language description of the target element. When the CSS selector fails and an LLM is configured, it resolves the element automatically. See LLM Fallback.

Full Workflow Example

# 1. Create a tab
curl -X POST http://localhost:3000/api/browser/tabs \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# 2. Fill a form (with LLM fallback per field)
curl -X POST http://localhost:3000/api/browser/tabs/0/fill \
  -H "Content-Type: application/json" \
  -d '{"fields": [
    {"selector": "input[name=email]", "value": "[email protected]",
     "llmFallback": "The email input field"},
    {"selector": "input[name=password]", "value": "secret123",
     "llmFallback": "The password input field"}
  ], "llmFallback": "Fill the login form"}'

# 3. Click submit (with LLM fallback)
curl -X POST http://localhost:3000/api/browser/tabs/0/click \
  -H "Content-Type: application/json" \
  -d '{"selector": "button[type=submit]",
       "llmFallback": "Click the login button"}'

# 4. Wait for result (with LLM fallback)
curl -X POST http://localhost:3000/api/browser/tabs/0/wait \
  -H "Content-Type: application/json" \
  -d '{"selector": ".success-message", "timeout": 10000,
       "llmFallback": "The success confirmation message"}'

# 5. Get page content
curl "http://localhost:3000/api/browser/tabs/0/source?type=text"

Silent-tab & surface-dependent operations

LumaBrowser is a single-window, single-visible-tab surface. A tab that is backgrounded (another tab is foregrounded) or was created with silent: true has no rendered display surface. A handful of operations require that surface and will fail on a hidden tab:

• GET /api/browser/tabs/:id/screenshot - fails with "Current display surface not available for capture". Foreground the tab first with POST /api/browser/tabs/:id/activate.
• POST /api/browser/tabs/:id/scroll - a scroll call returns successfully, but window.scrollY on a backgrounded tab doesn't always progress as a user-driven scroll would. Activate the tab first for deterministic scroll behavior.

For deterministic foregrounding in multi-tab automation, call POST /api/browser/tabs/:id/activate (or PATCH with {"action":"activate"}) immediately before any surface-dependent operation. Silent tabs cannot be activated; if you need to screenshot or scroll a tab, create it with silent: false.

Every selector-based browser command accepts an optional llmFallback parameter, a plain-English description of the target element. When a selector fails (or no selector is given at all), LumaBrowser runs your description through a three-stage pipeline that starts deterministic, replays an answer it already validated on this kind of page, and escalates to the LLM only when needed, validating what the LLM returns and remembering it for next time.

The Three-Stage Resolution Pipeline

This pipeline runs whenever a caller invokes any LLM-fallback path, REST llmFallback on click / wait / getElement / fill, the Selenium ai-description locator, /lumabyte/find, /lumabyte/click, the Puppeteer CDP methods Lumabyte.find and Lumabyte.click, and the MCP tools listed below. Every stage has one job, and most requests never reach the LLM at all.

Stage 1 - Deterministic match (0 tokens)

Before any LLM call, the resolver queries the live DOM with a battery of stable attribute strategies derived from your description:

• [data-testid] - both the slugified description and the raw string
• [aria-label]
• [placeholder]
• [data-role] - slugified description
• Exact visible-text match on interactable elements

If any strategy produces exactly one visible element, the resolver tries the action with it immediately. No LLM tokens spent, no latency added. Pages that expose stable semantic attributes resolve at native speed. If the action fails with that element, the resolver assumes it picked the wrong one and moves on to the next stage.

Stage 2 - Local resolution cache (0 tokens)

Next the resolver looks in its local resolution cache: earlier LLM (and vision) answers, stored as robust CSS selectors keyed by page URL pattern, action and normalized description. A cached selector is only replayed after a live check that it still names exactly one visible element with the same tag and similar text. Anything doubtful counts as a miss, and a cached selector that fails is dropped. This stage runs even when no LLM is configured.

• Stored in LumaBrowser's local settings database on your machine; nothing is shared or uploaded.
• Up to 2,000 entries, least recently used evicted first; entries expire after 30 days.
• On by default. Turn it off with the core.browser.resolutionCache.enabled setting.
• A response served from the cache carries resolvedBy: "cache".

Stage 3 - LLM with a live DOM snapshot

If both earlier stages miss, the LLM is prompted with a compact snapshot of up to 80 interactable elements from the actual current DOM. Each snapshot row carries the element's selector and tag plus id, data-testid, data-role, aria-label, role, name, type, placeholder, disabled, and visible text. The model is told to ground its answer strictly in that snapshot. The snapshot is the only page context; there is no stored page map.

Implication for dynamically-rendered content. The LLM sees what is in the DOM at the moment of the call. If your target element appears after a delay (lazy hydration, skeleton-swap, modal after click), trigger the delay before invoking wait_for with a description, or allow wait_for to poll until the element mounts. An element that is not in the DOM cannot be resolved.

Validate and retry. Before the action runs, the LLM's selector is validated against the live DOM and must match exactly one element (or exactly one visible element). If it matches zero or several, returns nothing usable, or repeats a selector that already failed, the reason is fed back into the prompt and the LLM gets one more attempt. Total LLM calls are capped at two per resolution, no runaway costs on a pathological page.

Remember. The validated element is captured as a robust selector before the action runs (a click may navigate away), and stored in the resolution cache once the action succeeds. The next call with the same description on the same kind of page resolves in Stage 2 without an LLM call.

Why This Is Better

• Free speed on well-built pages. Pages with stable data-testid, aria-label, placeholder, or data-role attributes resolve in Stage 1 with zero LLM cost and sub-millisecond latency.
• LLM grounded in current truth. The model sees a snapshot of the real DOM, not a stale dump; so dynamic, client-rendered, and single-page-app content work as long as the target is actually mounted at call time.
• Validated before acting. A hallucinated or ambiguous selector is rejected and retried before it touches the page, no silent misclicks on the wrong element.
• Self-warming cache. Every successful LLM resolution pays forward: routine workflows and scheduled tasks converge to zero-token replays from the local cache over time.

Prerequisites

• The selector-resolver LLM slot, or a globally configured primary LLM (Settings → LLM Providers). See LLM slot configuration below. Stages 1 and 2 work without any LLM.
• Works with the bundled local llama.cpp server, OpenAI-compatible endpoints, or Anthropic.

LLM slot: selector-resolver

LumaBrowser registers a dedicated core-scope LLM slot for this pipeline, labelled Selector Resolver (fast, low-latency model recommended). If the slot is not explicitly configured, the resolver falls back to your global active LLM.

Deployment recommendation. Configure this slot with a fast, low-latency model: Claude Haiku, Gemini Flash, GPT-4.1 mini, or a similar speed-tier option. Heavier reasoning models are overkill for selector resolution and measurably slow the action path. The temperature: 0 guarantee means a small model performs well here; there is no reasoning budget to spend.

Configure the slot under Settings → LLM Providers → Selector Resolver.

Supported Endpoints

Example: Click with Fallback

# Selector works → instant click, llmFallback is ignored
# Selector breaks → LLM resolves the right element automatically
curl -X POST http://localhost:3000/api/browser/tabs/0/click \
  -H "Content-Type: application/json" \
  -d '{
    "selector": "button.btn-primary.submit-form",
    "llmFallback": "Click submit"
  }'

# Response on LLM fallback success:
{
  "data": { "clicked": true, "resolvedSelector": "form button[type=submit]" },
  "success": true,
  "message": "Element clicked successfully (resolved by LLM fallback)"
}

Example: Click without Selector (LLM-Only)

# No selector at all; LLM resolves from description alone
curl -X POST http://localhost:3000/api/browser/tabs/0/click \
  -H "Content-Type: application/json" \
  -d '{
    "tabId": 0,
    "llmFallback": "Click the Sign In button in the navigation bar"
  }'

Example: Form Fill with Per-Field Fallback

curl -X POST http://localhost:3000/api/browser/tabs/0/fill \
  -H "Content-Type: application/json" \
  -d '{
    "fields": [
      {"selector": "input#email", "value": "[email protected]",
       "llmFallback": "The email address input"},
      {"selector": "input#pass", "value": "secret",
       "llmFallback": "The password input"},
      {"selector": "select#role", "value": "admin",
       "llmFallback": "The role dropdown"}
    ],
    "llmFallback": "Fill the registration form"
  }'

Debugging resolution (operator)

To trace why a specific description failed to resolve, launch LumaBrowser with the environment variable LLM_FALLBACK_DEBUG=1. The resolver writes verbose, stage-by-stage traces to stdout prefixed with [LlmFallback]: which stage answered, how many elements were in the DOM snapshot, what the LLM returned, and why validation accepted or rejected it. RESOLUTION_CACHE_DEBUG=1 traces the resolution cache on its own.

MCP Tools

The corresponding MCP tools also accept llmFallback: browser_click, browser_type, browser_wait_for, browser_scroll, browser_press_key, browser_get_element, browser_get_table, and per field on browser_fill_form. An element ref from browser_observe_page always wins over a description. When a vision grounding model is configured, browser_click also tries a vision click as a last resort. This means AI agents using the MCP server can provide their own fallback descriptions for resilient automation.

Connect Claude Code, Codex, OpenCode, Cline, Claude Desktop, or any MCP-compatible AI agent to LumaBrowser's built-in MCP server. Give your agents native browser capabilities without writing glue code.

Setup

From Settings (recommended). Connect your agent writes the LumaBrowser entry into Claude Code, Codex, OpenCode and Cline for you. For any other client, Export MCP Configuration saves a luma-mcp-config.json with the exact entry for your install, ready to merge into the client's config.

By hand. The MCP server is a small stdio script that ships with the app and runs under Node.js, so Node must be installed for the node command. Add this to your client's MCP config (for Claude Desktop, claude_desktop_config.json: %APPDATA%\Claude\ on Windows, ~/Library/Application Support/Claude/ on macOS):

{
  "mcpServers": {
    "luma-browser": {
      "command": "node",
      "args": ["<install dir>/resources/app.asar.unpacked/mcp-server.js"],
      "env": { "LUMA_API_PORT": "3000" }
    }
  }
}

Typical script paths:

• Windows (per-user install): %LOCALAPPDATA%\Programs\LumaBrowser\resources\app.asar.unpacked\mcp-server.js
• macOS: /Applications/LumaBrowser.app/Contents/Resources/app.asar.unpacked/mcp-server.js

If you installed somewhere else (or launch through npx lumabrowser), use Export MCP Configuration to get the right path. Set LUMA_API_PORT to match core.apiPort if you changed it. The MCP server talks to the REST API and starts LumaBrowser if it is not already running.

Available MCP Tools

Browser Automation

Extension Tools

Example: Claude Browsing the Web

User: "Go to Hacker News and tell me the top 3 stories."

Claude uses MCP tools:
  → browser_navigate({ tabId: 0, url: "https://news.ycombinator.com" })
  → browser_get_source({ tabId: 0, type: "text" })
  → "Here are the top 3 stories: ..."

Monitor HTTP traffic using Chrome DevTools Protocol (CDP) and forward matched responses to webhooks. No proxy setup or root certificates required.

How It Works

Network Watcher uses CDP to intercept responses at the browser level. You define URL patterns and a webhook destination; when traffic matches, the request is forwarded to your endpoint.

API Endpoints

Create a Watcher

curl -X POST http://localhost:3000/api/watchers \
  -H "Content-Type: application/json" \
  -d '{
    "urlPattern": "*api.example.com/v1/orders*",
    "sendTo": "https://your-backend.com/webhook",
    "note": "Capture order API calls",
    "method": "POST",
    "captureHeaders": true,
    "captureBody": true
  }'

Webhook Payload

When a matching request is intercepted, this payload is POSTed to your sendTo URL. Note that request is a composite envelope: it carries the outbound request metadata and nests the captured response inside it.

{
  "watcherId": "watcher_1711632896000_abc",
  "note":      "Capture order API calls",
  "timestamp": "2026-03-28T12:34:56.000Z",
  "request": {
    "url":       "https://api.example.com/v1/orders",
    "method":    "POST",
    "timestamp": "2026-03-28T12:34:55.980Z",

    "request": {
      "headers": { "content-type": "application/json", "authorization": "Bearer ..." }
    },

    "response": {
      "status":        200,
      "statusText":    "OK",
      "mimeType":      "application/json",
      "headers":       { "content-type": "application/json" },
      "body":          "eyJvcmRlcklkIjoiMTIzNDUifQ==",
      "base64Encoded": true
    }
  }
}

Where to find each field:

• Request headers: request.request.headers (double-nested, the outer request is the capture envelope, the inner request is the browser's outbound HTTP request).
• Response status / body / headers: request.response.* (not at the top level).
• Encoding: bodies from non-text MIME types are base64-encoded; check request.response.base64Encoded before decoding.

Watcher Configuration

Toggle a watcher

POST /api/watchers/:id/toggle requires a body; it sets the enabled flag explicitly rather than flipping it. A body-less call fails with a 500. Always supply enabled:

POST /api/watchers/watcher_xxx/toggle
{"enabled": true}   // or false

Updating a watcher (PATCH caveat)

PATCH /api/watchers/:id behaves as a full-row update and will reject partial bodies that omit required columns. For example, a PATCH with only {"note": "..."} fails with "NetworkWatcher: urlPattern is required". Re-send every required field (urlPattern, sendTo) in the update body, or use the toggle endpoint for enable/disable changes.

Schedule recurring LLM-driven browser automation tasks. Each execution runs an agentic loop (up to 10 tool-call iterations) with full access to the browser automation API.

API Endpoints

Create a Scheduled Task

The request body is camelCase. The response (below) is snake_case and uses 0/1 for enabled. This asymmetry is documented rather than fixed to keep existing SDKs working; model it explicitly in your client.

curl -X POST http://localhost:3000/api/timed-tasks \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Check Product Prices",
    "requestPrompt": "Navigate to https://store.example.com, find the price of Widget X, and return it as JSON",
    "responsePrompt": "Format the result as {\"product\": \"...\", \"price\": \"...\"}",
    "repeatInterval": 3600000,
    "webhookUrl": "https://my-backend.com/prices",
    "enabled": true
  }'

Request vs. response shape

Request (camelCase, enabled is boolean):

{
  "name":           "Check Product Prices",
  "requestPrompt":  "...",
  "responsePrompt": "...",
  "repeatInterval": 3600000,
  "webhookUrl":     "https://my-backend.com/prices",
  "enabled":        true
}

Response (snake_case, enabled is 0/1):

{
  "success": true,
  "task": {
    "id":               "task_1776605796275_iw0ooo",
    "name":             "Check Product Prices",
    "request_prompt":   "...",
    "response_prompt":  "...",
    "repeat_interval":  3600000,
    "webhook_url":      "https://my-backend.com/prices",
    "enabled":          1,
    "last_run":         null,
    "next_run":         null,
    "created_at":       "2026-04-19 12:34:56"
  }
}

PATCH semantics - read this before updating a task

PATCH /api/timed-tasks/:id currently behaves as PUT: it performs a full row update and nulls any column you don't re-send. A partial-looking PATCH fails loudly on required columns:

PATCH /api/timed-tasks/task_xxx
{"name":"new-name"}

→ 400 NOT NULL constraint failed: timed_tasks.request_prompt

For enabling or disabling a task, use the dedicated endpoints:

# Enable - no body, no "are all required fields set?" gotcha
curl -X POST http://localhost:3000/api/timed-tasks/task_xxx/enable

# Disable
curl -X POST http://localhost:3000/api/timed-tasks/task_xxx/disable

# Response for both
{
  "success": true,
  "task":    { /* full snake_case task object */ },
  "message": "Task enabled"    // or "Task disabled"
}

For other edits, re-send the full task body; every NOT NULL column (name, request_prompt, response_prompt, repeat_interval) must be present. The same PATCH-as-PUT behavior applies to PATCH /api/watchers/:id.

Execution Lifecycle

1. Task fires at the configured interval
2. A browser tab is pre-created
3. LLM receives the prompt with available browser tools
4. Agentic loop: LLM calls tools, gets results, decides next action (up to 10 iterations)
5. Final result is stored and forwarded to the webhook URL

Monitors web pages for content changes by extracting text on a schedule, comparing SHA-256 checksums against the prior snapshot, and forwarding changes to a webhook and/or desktop notification. Includes an interactive in-page element picker so you can scope a monitor to specific DOM elements rather than the whole noisy page. See the Page Change Detector overview for the full feature walkthrough.

API Endpoints

Create a monitor

curl -X POST http://localhost:3000/api/page-change-detector \
  -H "Content-Type: application/json" \
  -d '{
    "name":                  "Acme Pricing Page",
    "url":                   "https://acme.com/pricing",
    "checkIntervalMs":       300000,
    "intervalJitterPercent": 10,
    "webhookUrl":            "https://hooks.slack.com/services/XXX/YYY/ZZZ",
    "desktopNotifications":  true,
    "enabled":               true
  }'

Monitor fields

Webhook payload

When a change is detected and webhookUrl is set, this JSON is POSTed with Content-Type: application/json (10 s timeout):

{
  "monitorId":    "mon_1714742312000_a4b1k2",
  "monitorName":  "Acme Pricing Page",
  "url":          "https://acme.com/pricing",
  "timestamp":    "2026-05-03T17:22:08.412Z",
  "checksum":     "9f3c1b2e... (sha256 of new extracted text)",
  "prevChecksum": "1a8d04bf... (or null on the first detected change)",
  "diffSummary":  "+2 / -1\n+ $19.99 - In stock\n- $24.99 - Out of stock",
  "textPreview":  "First 2000 characters of the new extracted text…",
  "textLength":   4821,
  "changeCount":  17
}

• diffSummary is multi-line: line 0 is the stats header (+N / -M, or whitespace / formatting only), followed by up to four added (+ ) and four removed (- ) snippets, each truncated to 140 characters.
• For the first detected change on a fresh monitor, prevChecksum may be null; treat the first webhook as a baseline.
• If extraction fails on a cycle (page error, network blip), the cycle is logged and skipped. No phantom diff is recorded.

Element picker (POST /:id/pick)

Opens the monitor's URL in a visible foreground tab (silent tabs are not reused; the user has to be able to see the page) and injects an in-page picker. The user clicks elements to select; clicks again to deselect; presses Done, Cancel, or Esc.

curl -X POST http://localhost:3000/api/page-change-detector/mon_xxx/pick

# success - user clicked Done
{ "success": true, "cancelled": false,
  "selectors": ["#price-tile", ".stock-badge"],
  "monitor": { /* updated monitor with new selectors */ } }

# user cancelled
{ "success": true, "cancelled": true }

The endpoint blocks until the picker resolves. Existing selectors are surfaced to the picker so the user sees what's already chosen and can refine without starting from scratch.

History (GET /:id/history)

Two response shapes depending on query params:

Each history entry includes id, monitor_id, checksum, changed (0/1), checked_at (ISO timestamp), text_length, text_preview (first 500 chars), and diff_summary.

The Hub extension gathers your calendars, a conversation queue (intercepted chat and mail notifications) and a unified task board into one place. Its REST routes under /api/hub let an automation such as n8n push queue items, enrich threads, and read or change the board. Agents get the same surface as hub_* MCP tools.

Authentication

Every route except the OAuth callback needs the Hub token, sent as Authorization: Bearer <hub token> or X-Hub-Token: <hub token>; otherwise 401. The token (hub_ plus 48 hex characters) is created on first use and can be rotated from the Hub settings tab. It is separate from the app-wide API key, so use X-Hub-Token when the gateway's API key already occupies the Authorization header.

API Endpoints

Replies are { success: true, ...payload }; an error is a 400 with error. When a ClickUp task's target column has no linked status in its list, POST /tasks answers 400 and /move answers moved: false, both with needsStatus and statuses; send the request again with status.

Example: push a queue item

curl -X POST http://localhost:3000/api/hub/queue \
  -H "Content-Type: application/json" \
  -H "X-Hub-Token: hub_..." \
  -d '{
    "app": "slack",
    "threadKey": "C024BE91L-1712345678.000100",
    "title": "Deploy question from Dana",
    "summary": "Asks whether Friday release is still on",
    "priority": "high"
  }'

A built-in LLM-powered agent that can see and control the browser. Use it interactively via the sidebar, or programmatically via the REST API and MCP tool. Give it a single natural language prompt and it autonomously navigates, clicks, fills forms, and extracts data, returning a structured result.

Capabilities

• Agentic tool execution loop (up to 15 iterations per run, configurable)
• Full access to all browser automation tools
• Self-healing element targeting through the shared resolution pipeline and its local cache
• Headless REST API for programmatic use
• MCP tool (ai_chat_run) for agent-to-agent orchestration
• Optional screenshot capture of final page state
• Tab lifecycle management (auto-create, reuse existing, auto-close)
• Tool allowlisting for read-only or restricted runs

REST API Endpoint

Request

POST /api/ai-chat/run
{
  "prompt": "Log into acme.com/login with [email protected] / demo123,
             go to settings, change company name to 'Acme Industries'",
  "tabId": 0,                        // optional - use existing tab
  "autoCloseTab": true,               // optional, default: true
  "includeScreenshot": false,          // optional, default: false
  "maxIterations": 15,                 // optional, default: 15
  "timeout": 120000,                   // optional, default: 120000 (ms)
  "tools": ["navigate", "click", ...], // optional - restrict available tools
  "systemPromptAppend": "Return JSON"  // optional - extra instructions
}

Request Parameters

Response

{
  "success": true,
  "summary": "Logged in and updated company name to Acme Industries",
  "finalResponse": "Done! I logged into acme.com, navigated to settings...",
  "tabId": 0,
  "iterations": 4,
  "durationMs": 18420,
  "toolCalls": [
    { "tool": "navigate", "params": {"tabId": 0, "url": "..."}, "durationMs": 1200 },
    { "tool": "fill_form", "params": {"tabId": 0, "fields": [...]}, "durationMs": 340 },
    { "tool": "click", "params": {"tabId": 0, "selector": "..."}, "durationMs": 890 },
    { "tool": "wait_for", "params": {"tabId": 0, "selector": "..."}, "durationMs": 1100 }
  ],
  "screenshot": "data:image/png;base64,..."
}

Response Fields

Note on includeScreenshot

includeScreenshot: true authorizes the agent to take a screenshot; it does not force one. If your prompt doesn't imply visual verification, the agent will often finish without ever calling the screenshot tool, and the top-level screenshot field will be absent.

To reliably capture a final screenshot, add an explicit instruction to the prompt:

{
  "prompt": "Log into acme.com, go to the billing page, then TAKE A SCREENSHOT of the final state.",
  "includeScreenshot": true
}

If you only want the bytes and don't care about visual verification during the run, call GET /api/browser/tabs/:id/screenshot directly after the run completes (set autoCloseTab: false to keep the tab open). That endpoint returns in under a second for a foregrounded tab.

Tool result shapes

Each entry in toolCalls[] records the invocation metadata:

{
  "tool":       "screenshot",
  "params":     { "tabId": 23 },
  "durationMs": 66,
  "success":    true,
  "error":      null
}

The actual payload produced by each tool lives under steps[N].toolResult (or equivalently steps[N].toolResult.data). Shapes by tool:

If you need screenshot bytes that an agent captured mid-run, extract them from steps[].toolResult.data.screenshot rather than the top-level screenshot field.

Click tool schema (what the model sees)

The AI chat agent is instructed that click takes a real CSS selector in selector, and that text is an optional visible-text filter for disambiguating when the selector matches multiple elements. If you're routing your own LLM into LumaBrowser's click tool, align your system prompt to the same shape:

# Single unique match - selector alone
{ "tool": "click", "params": { "tabId": 0, "selector": "#btn-simple" } }

# Multiple matches - use text to pick
{ "tool": "click", "params": { "tabId": 0, "selector": "button", "text": "Sign In" } }

selector must be a CSS selector string, not a description, not the literal word "css".

Example: Scrape Data (Read-Only)

curl -X POST http://localhost:3000/api/ai-chat/run \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Go to news.ycombinator.com and return the top 5 story titles as JSON",
    "tools": ["navigate", "get_source", "wait_for", "scroll"],
    "systemPromptAppend": "Return only a JSON array of strings",
    "maxIterations": 5
  }'

Example: Reuse a Tab

# Step 1: Create a tab and keep it open
curl -X POST http://localhost:3000/api/ai-chat/run \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Navigate to acme.com/dashboard and log in with [email protected] / demo123",
    "autoCloseTab": false
  }'
# → returns { "tabId": 0, ... }

# Step 2: Continue on the same tab
curl -X POST http://localhost:3000/api/ai-chat/run \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Now go to the settings page and change the timezone to UTC",
    "tabId": 0,
    "autoCloseTab": false,
    "includeScreenshot": true
  }'

MCP Tool

The same functionality is available as an MCP tool: ai_chat_run. It accepts all the same parameters and returns the same result shape. This enables agent-to-agent orchestration; an outer AI agent can delegate complex browser tasks to LumaBrowser's built-in agent.

Available Browser Tools

The agent can use: get_tabs, create_tab, navigate, observe_page, click, click_at, locate, type, fill_form, select_option, set_date, set_slider, press_key, scroll, wait_for, get_element, get_source, extract_data, collect_list, screenshot. These are the chat agent's names for the same actions the MCP server exposes as browser_* tools.

AI Chat can produce artifacts, standalone documents rendered live beside the conversation. The model creates them with chat tools; they are versioned and individually addressable.

Artifact types

Chat tools

Validation

Code artifacts are auto-linted on create/edit (JavaScript/TypeScript, JSON, PowerShell); live modules pass a fatal JS syntax gate before rendering plus a non-fatal lint pass; the model is instructed to fix problems before declaring the work done.

Versioning

Every edit creates a new row sharing the original root_id with an incremented version; the chat lists one entry per artifact with a version count; full history is retained; deleting an artifact removes the whole version chain.

Accessing artifacts

Served by the local REST gateway:

For LAN sharing (paired clients, Bearer token):

Define named sub-agents, each with its own system prompt, a curated tool allowlist, and an optional pinned model. The main chat agent can delegate to them, or you can call them over REST. Base path: /api/ext/agent-manager.

REST API

Example: Create an agent

curl -X POST http://127.0.0.1:3000/api/ext/agent-manager/agents \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Research Agent",
    "description": "Web research",
    "systemPrompt": "You are an expert researcher. Cite sources.",
    "tools": ["get_source", "click"],
    "modelRef": null
  }'

Example: Run an agent

curl -X POST http://127.0.0.1:3000/api/ext/agent-manager/agents/research-agent/chat \
  -H "Content-Type: application/json" \
  -d '{ "message": "Summarize today AI news", "stream": false }'

MCP tools

Lets any MCP host or the in-app agent delegate to a sub-agent.

Per-agent knowledge base

Any agent can be given its own private set of documents. When an agent has documents, it automatically gains the search_knowledge_base tool (scoped to that agent only) and a system-prompt note instructing it to search before answering. See the Knowledge Base section and the marketing overview.

LumaBrowser can ground an agent's answers in your own documents. Files are parsed, chunked, and indexed in a local SQLite database (FTS5 full-text search, BM25 ranking); retrieval runs entirely on-device with no embeddings API and nothing uploaded. A marketing overview lives at /knowledge-base.

Ingestion

• Accepted formats: .pdf, .txt, .md, .markdown, .html, .htm. (DOCX and URL ingestion are not supported.)
• Chunking: recursive-separator split, greedily merged to ~500-token passages with ~64-token overlap. PDFs are read page by page so citations can reference a page number.
• De-duplication: per-scope by content hash (SHA-256), so re-adding the same file is a no-op.

Scopes

Every document and chunk carries a scope. Per-agent knowledge bases use scope agent:<id> and are fully isolated from one another and from the default chat scope. The shipped end-user path for adding documents is the Agents tab: edit an agent and click Add documents (native file picker).

Agent tool

Example: add documents to an agent, then ask

# 1. Ingest two files into an agent's knowledge base
curl -X POST http://127.0.0.1:3000/api/ext/agent-manager/agents/support-agent/kb \
  -H "Content-Type: application/json" \
  -d '{ "paths": ["C:/docs/billing-policy.pdf", "C:/docs/terms-2026.md"] }'

# 2. Run the agent - it now auto-searches its documents before answering
curl -X POST http://127.0.0.1:3000/api/ext/agent-manager/agents/support-agent/chat \
  -H "Content-Type: application/json" \
  -d '{ "message": "What is our refund window for annual plans?", "stream": false }'

A bundled image server (built on stable-diffusion.cpp) runs text-to-image and image editing locally. Models download on demand from Hugging Face; a generation model and an edit model can stay resident at once. A marketing overview lives at /local-image-generation; see also /local-ai for setup.

What it supports

• Text-to-image
• Image-to-image
• Inpainting (mask)
• Reference/identity images for subject continuity

Curated models

An optional Anima anime add-on is available, and you can bring your own single-file .safetensors/.gguf or import a Hugging Face repo.

From AI Chat

The agent has generate_image and edit_image tools; results return as image artifacts. Runs on NVIDIA (CUDA), cross-vendor GPUs (Vulkan), and Apple Silicon.

LoRAs & speed presets

Drop .safetensors LoRAs into the shared loras/ directory beside your models; they're passed to the image server as --lora-model-dir and fold in at load time. Each still-image (SD/SDXL-family) model row has a “LoRAs / speed” button that opens a modal to attach LoRAs or import one.

• Acceleration LoRAs (DMD2, SDXL-Lightning, Hyper) - tick Apply distilled speed preset and the model flips to steps=8, cfg=1.0, sampler euler so the speedup lands.
• Style LoRAs - any style .safetensors in the library can be attached with a per-LoRA weight.

Attaching or changing LoRAs is a launch-argument change, so an already-running slot for that model is stopped and the next generation cold-starts with the new LoRA set. IPCs: core.imageServer.listLoras, importLora, setModelLoras.

Roleplay is an optional chat mode for illustrated, interactive roleplay: a local language model narrates while a local image model paints a reaction image on each turn, keeping characters consistent across scenes. A marketing overview lives at /roleplay. It ships as an optional add-on (not bundled) - enable it from the Extensions area.

How it works

1. Setup - pick Roleplay in the mode picker; a form collects the scenario, characters, scenes, and options.
2. Play - write in prose; the model narrates the scene forward.
3. Staging - after each turn, a wardrobe pass decides each character's current outfit (tracking head/face accessories separately) and a director pass chooses pose, framing, and focus.
4. Image - a reaction image is composed and posted inline, with live mode:progress / mode:image-stage previews as the shot is staged and refined.

Consistency techniques

• Per-character seed + reference portrait for identity across scenes.
• Pose-variant composite - keeps the established face and grafts it onto a re-posed figure rather than re-rolling (which loses identity), then composites into the scene with matched lighting.
• 157-expression emotion catalog (9 categories) with masked face inpainting, so only the expression repaints while the rest of the character holds steady.
• Multi-character composition - stages up to two characters into one scene with shared ambient light.

Models

Roleplay pairs a base illustration model (e.g. an Anima-family Danbooru-tag model) with an editing model (e.g. Qwen-Image-Edit) that places and harmonizes characters into scenes. Both are chosen in setup and default to the image server's defaults.

Code Mode is an AI build agent that turns a plain-English request into a real, installable LumaBrowser extension. Open it via the Extensions panel's "Vibe" button or the "Code" chat mode. See also /code-mode and /extensions.

Workflow

1. Start - describe the extension.
2. Plan - agent outlines manifest.js, main.js, optional UI/routes.
3. Build - writes each file with live validation.
4. Validate - syntax-checked as written.
5. Install - validates + activates live, no restart.
6. Use immediately.

Build tools

These run in a sandboxed per-build workspace.

Built extensions are real user-installed extensions you can keep, disable, or permanently delete (first-party bundled ones can only be disabled).

Code a project

Alongside “Vibe” (build an extension), Code Mode has a “Code a project” entry that points the agent at an existing folder on disk. It opens the directory read/write with path-traversal guards (the folder is never created or deleted for you) and exposes project-oriented tools: project_overview, read_file, grep, find, list_dir, edit_file (surgical exact-then-fuzzy edits that preserve line endings), and write_file. Longer tool output is truncated to keep the context lean, and larger jobs can fan work out across sub-runs bounded by the model server's real decode-slot budget. Search and the project map are plain filesystem operations - no external indexer or embeddings.

Beyond exposing its own MCP server, LumaBrowser can act as an MCP client, connect to external MCP servers (stdio, HTTP, or SSE), discover their tools live, and merge them into one tool surface. External tools are namespaced as mcp__<server-slug>__<tool-name>. Base path: /api/ext/mcp-connector.

REST API

Server config

Example: Add a stdio server

curl -X POST http://127.0.0.1:3000/api/ext/mcp-connector/servers \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Files",
    "transport": "stdio",
    "command": "npx",
    "args": ["@modelcontextprotocol/server-filesystem", "/data"]
  }'

Aggregated MCP REST endpoints

These core endpoints expose ALL tools (built-in + connected external):

LAN Sharing

Share this machine's local LLM and image-generation servers (and agent tools) to other devices on your network. Clients pair with a 4-digit PIN; an optional browser PWA client is available. Toggle which resources are shared (local LLM, remote LLMs, image generation, image editing).

GPU & VRAM placement

A drag-and-drop layout to pin the LLM, image-generation, and image-edit models to specific GPUs (or system RAM), with an optional single-GPU "singularity" hotswap pool that evicts the previous model before loading the next. Auto-arrange returns to automatic allocation; a measurement test records peak VRAM/RAM per model so fit bars are accurate.

Cache clearing

Clear the browser's HTTP/disk cache for all tab sessions at once from the settings menu.

Activity log

An opt-in audit trail of timed action spans per extension/service, with configurable retention (days + max rows) and per-caller toggles; off by default.

LumaBrowser can run a language model entirely on your machine, with no API key and nothing leaving the device. The LLM Server is a managed llama.cpp server: a guided Easy Setup wizard reads your hardware, recommends a curated GGUF model that fits, resumably downloads it, starts the server, and drops you into a built-in chat. A marketing overview lives at /local-ai; this section is the implementation reference.

Easy Setup flow

Setup runs on first launch (inside the onboarding wizard) and is also reachable any time from the LLM tab via the ✨ Easy Setup button. It asks three questions, then turns the answers plus detected hardware into a concrete pick:

The wizard then, in order: checks for the recommended llama.cpp runtime and installs it only if missing → resumably downloads the model → persists defaults (runtimeId, modelPath, contextSize, kvCacheType) → starts the server → opens chat mode. Each step shows progress; the model download can be canceled and resumed later.

Curated model catalog

The picker chooses from a curated list of verified single-file GGUF repositories (bartowski quants on Hugging Face, downloaded over public /resolve/ URLs, no account needed). Every entry is offered in Q4_K_M, Q6_K, and Q8_0.

Approximate download sizes (Q4_K_M): Gemma 4 E2B ~3.2 GB, E4B ~5.0 GB, Gemma 4 26B-A4B ~15.9 GB, Qwen 3.6 27B ~16.3 GB, Gemma 4 31B ~18.3 GB, Qwen 3.6 35B-A3B ~19.9 GB. Q6_K and Q8_0 are progressively larger. The exact size is read from the HTTP Content-Length at download time; catalog figures are estimates used only for tiering and the pre-download display.

How the recommender sizes your setup

The recommender is a pure heuristic that runs before anything downloads, so it can recommend instantly. Logic:

• Runtime: llama-cpp-cuda12 if a CUDA-capable NVIDIA driver is present, else llama-cpp-vulkan if any GPU is present, else llama-cpp-cpu.
• Model: the most capable catalog model for your use case whose best-quality quant fully fits usable VRAM. If nothing fully fits and you chose a patient/balanced speed, it allows the largest model whose weights fit in RAM (CPU or partial GPU offload). Otherwise it falls back to the smallest model on CPU, always works, just modest.
• Context: starts from your preference (4096 / 16384 / 49152), clamped to the model's max. On a full-GPU pick it sizes context honestly from real weight size + KV cache + a fixed compute overhead; so on a roomy GPU you get a genuinely large context instead of a collapsed 4K floor. If the f16 KV cache won't fit, it switches the KV cache to q8_0 (about half the VRAM) before shrinking context.

The recommendation comes back with a plain-language rationale string and, when relevant, a warnId (cpu-only or partial-offload) so the UI can warn that the pick will be slow.

Bundled runtimes

Inference runtimes are acquired and managed for you. The runtime catalog is declarative; the detector looks in the managed install directory first, then the host PATH.

Power users: an empirical fit test sweeps real context rungs (8K, 16K, 32K, 64K, 128K, 256K) and records measured tok/s and VRAM per model. The recommender is the instant first guess; the fit test is the slow, accurate confirmation. Per-model context options in the chat picker show measured rungs when a fit result exists and an estimated fit otherwise.

Resumable downloads

Model files are 3–30 GB, so the downloader keeps a sibling .partial file and resumes with an HTTP Range request. If a download is interrupted (connection drop, app quit, explicit cancel), the partial file is kept and the next attempt continues from the saved offset, nothing restarts from zero. A server that ignores the range header (responds 200 instead of 206) safely restarts; a range past EOF (416) means the partial is already complete.

Bring your own GGUF

The setup recommendation is a default, not a cage. The Advanced field accepts any single-file GGUF, a full Hugging Face resolve URL or an owner/repo/file.gguf shorthand, and the same resumable downloader handles it.

# Either form works in the Easy Setup "Advanced" field:
https://huggingface.co/owner/repo/resolve/main/model.gguf
owner/repo/model.gguf

Chat features

Once a model is running, the LLM tab is a full chat surface backed by SQLite conversation history:

• Token streaming - responses stream as generated, with scroll-pause / auto-scroll.
• Live artifact panel - ask the model to build a page, chart, diagram, or document and it renders in a docked panel beside the chat as it is written (the partial create_artifact content is sniffed out of the token stream), not pasted back as raw markup. Pop out to a real browser tab from the panel.
• Regenerate with variants - regenerate any assistant message; prior takes are kept as variants you can page between.
• Agentic Tools toggle - turn on Tools and the same model drives the browser (navigate, click, fill, extract) via the ai-chat agent loop. A lenient tool-call parser recognizes calls that small/quantized models emit as a bare or ```json-fenced object, not just the strict ```tool fence.
• Unified model picker - switch between the local model and any configured remote provider (Anthropic / OpenAI-compatible) per conversation. Display names are normalized from the file stem (curated catalog labels win; otherwise a heuristic prettifier, e.g. google_gemma-4-31B-it-Q8_0 → “Gemma 4 31B”).

Local OpenAI / Anthropic-compatible API

An optional local endpoint serves the running chat model to other tools on this machine, in both the OpenAI and Anthropic formats. It is off by default; turn it on in Settings. When on it binds http://127.0.0.1:8317 only (port configurable), needs no key, and never starts a model by itself: start one in LumaBrowser first.

Settings → Connect your agent can point Claude Code, Codex, OpenCode and Cline at this endpoint for you. It is separate from LAN Sharing, which is the PIN-paired network surface.

Privacy

When a local model is selected, prompts and responses are processed entirely by the on-device llama.cpp server. There is no inference provider in the loop and no per-token cost. The app ships no third-party analytics or crash reporting; the only check-in is a lightweight app version ping, see /privacy, which never includes prompt or response content.

Drive LumaBrowser from any Selenium client, Python, Java, C#, Ruby, JavaScript, with zero code changes to your existing test suite. The selenium-driver extension embeds a W3C WebDriver Level 2 / ChromeDriver-compatible HTTP server inside LumaBrowser. Point RemoteWebDriver at it and the current browser tabs become your automation target, as if it were Chrome plus ChromeDriver.

The distinctive addition: LLM-powered selector fallback. When a CSS or XPath locator fails to match, or a click is intercepted, the driver can re-resolve the element from a natural-language description and retry automatically. Brittle selectors stop breaking builds.

Defaults

Field naming: The settings endpoint accepts urlPrefix on write; the status endpoint returns the same value as prefix. Treat them as the same field under two names.

Starting the Driver

Two ways to start the WebDriver server:

1. Settings tab. Open Settings → Selenium, check “Start automatically when LumaBrowser launches”, then click Save & Start.

2. REST API. Use LumaBrowser's existing API gateway on port 3000:

# Start the WebDriver server
curl -X POST http://localhost:3000/api/selenium/start

# Check status
curl http://localhost:3000/api/selenium/status

# Change port to 4444 and expose on the network
curl -X POST http://localhost:3000/api/selenium/settings \
  -H "Content-Type: application/json" \
  -d '{"host": "0.0.0.0", "port": 4444}'

Response shapes: /start vs /status

These two endpoints return different shapes today. If you need the full runtime description after a start, call /status after /start.

# POST /api/selenium/start - minimal
{
  "success": true,
  "already": false,   // true if the server was already running
  "port":    9515
}

# GET /api/selenium/status - complete
{
  "running":  true,
  "host":     "127.0.0.1",
  "port":     9515,
  "prefix":   "",          // urlPrefix as configured; empty by default
  "sessions": 0,           // active Selenium sessions
  "settings": { "host": "127.0.0.1", "port": 9515,
                "urlPrefix": "", "autostart": false, ... }
}

Construct the server root as http://{host}:{port}{prefix}. With default settings, that's http://127.0.0.1:9515 with no trailing path.

Python Example

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options

opts = Options()
# Opt into LumaBrowser's LLM fallback for this session
opts.set_capability("lumabyte:llmFallback", {
    "enabled": True,
    "onFindFail": True,
    "onClickIntercepted": True,
})

driver = webdriver.Remote("http://127.0.0.1:9515", options=opts)
driver.get("https://example.com")

# Standard Selenium - works exactly as you'd expect
title = driver.find_element(By.CSS_SELECTOR, "h1").text

# LumaBrowser extension: find an element by natural-language description
el = driver.execute(
    "lumabyte:find",
    {"description": "the main sign-up button near the top"},
)

driver.quit()

LLM Fallback - Four Override Points

The fallback system intercepts WebDriver at four points. Each is independently controllable via session capabilities.

1. Custom locator strategy: "ai-description"

Send a Find Element request whose using is "ai-description". No CSS needed; the LLM resolves the selector from your description.

POST /session/{id}/element
{ "using": "ai-description", "value": "the blue submit button" }

2. Auto-retry on zero matches

When a standard CSS or XPath selector matches nothing and the request included a lumabyte:description hint and fallback is enabled, the driver re-resolves via the LLM and retries once.

3. Click re-resolve

When element.click() fails because the click was intercepted or the element went stale, the driver re-resolves the element's original description and retries.

4. Vendor extension commands

LumaBrowser adds a namespaced command surface at /session/{id}/lumabyte/…:

POST /session/{id}/lumabyte/click - semantic click with optional selector hint

The click endpoint requires a description and accepts an optional selector hint. If both are provided, the driver tries the selector directly first and only falls through to description-based resolution if the selector fails, giving you the speed of a CSS click with the resilience of natural-language fallback.

The response body identifies which path won:

POST /session/abc123/lumabyte/click
{ "description": "the add to cart button", "selector": "button.buy-now" }

---

200 OK
{
  "value": {
    "clicked": true,
    "resolvedSelector": "button.buy-now",
    "resolvedVia": "selector"
  }
}

Session Capabilities

Set these under alwaysMatch or firstMatch when creating a session:

After session creation, the driver returns response capabilities that tell the client what's available:

LLM Slot

All Selenium LLM-fallback paths, the ai-description locator, /lumabyte/find, /lumabyte/click, and auto-retry on zero matches, route through the shared core-scope selector-resolver slot. Configure a fast, low-latency model there and every description-based Selenium call benefits automatically. If the slot is unconfigured, the resolver falls back to your global active LLM.

Known v1 Limitations

• Frame switching, shadow-DOM subqueries, async execute_async_script, and element-scoped screenshot cropping return unsupported operation.
• Keyboard special-key W3C PUA codepoints pass through as literal characters.
• POST /session/:id/lumabyte/cdp/execute with Page.captureScreenshot can hang past 100 s when the target tab is backgrounded. The non-Selenium GET /api/browser/tabs/:id/screenshot returns in under a second for a foregrounded tab; prefer it for routine screenshot work and reach for the CDP path only when you genuinely need a CDP-specific capture flag (clip, quality, full page, etc.).

Prefer CDP directly? If your client speaks Chrome DevTools Protocol natively (Puppeteer, Playwright via connectOverCDP, chrome-remote-interface, Playwright MCP, any cloud-browser-style connector) you can skip the WebDriver hop and attach to the CDP Driver on port 9222. Both drivers share the same LLM selector-resolution pipeline, so picking a protocol is a stylistic choice, not a capability tradeoff. See Selenium vs Puppeteer vs Playwright for a side-by-side.

Drive LumaBrowser from any Chrome DevTools Protocol client, Puppeteer, Playwright (via chromium.connectOverCDP), puppeteer-core, chrome-remote-interface, Playwright MCP, and the cloud-browser-style connectors that Browserless, Browserbase, and Steel.dev standardized on, by attaching to the built-in CDP server on port 9222. The cdp-driver extension boots a standards-compliant CDP WebSocket server plus the /json/* HTTP bootstrap endpoints every CDP client library uses for discovery.

Under the hood, the server is a translation layer rather than a from-scratch reimplementation of CDP: it owns the Browser, Target, and custom Lumabyte.* domains, and proxies every other command (Page, Runtime, DOM, Input, Network, Fetch, Emulation, Storage, Log, and friends, roughly 290 commands) straight through Electron's webContents.debugger to Chromium. You get the full native protocol surface for free.

The distinctive addition, identical in behavior to the Selenium driver: LLM-powered selector fallback exposed as the custom Lumabyte.* CDP domain. Any CDP session can resolve elements from a natural-language description via Lumabyte.find / Lumabyte.click, and the same orchestrator (deterministic match → local resolution cache → validated LLM attempt) runs for every client library.

Which clients work?

This is one endpoint, many clients. The protocol on the wire is pure CDP; the library you pick is a stylistic choice.

Dedicated automation tabs

Unlike the Selenium driver (which drives whichever tab is active), the CDP driver creates its own tabs with kind: 'cdp' and only those tabs surface as CDP targets. Automation-owned tabs are rendered in the tab strip with a purple accent and a CDP badge so you can watch the session drive them live. This is a deliberate design choice: webContents.debugger.attach is an exclusive lock, so isolating automation-kind tabs lets the Network Interceptor, AI Chat browsing tools, and other subsystems keep operating on the user's regular tabs without conflict.

Defaults

Starting the Driver

Two ways to start the CDP server:

1. Settings tab. Open Settings → CDP Driver, check “Start automatically when LumaBrowser launches”, then click Save & Start.

2. REST API. Management endpoints live on LumaBrowser's existing API gateway (port 3000), separate from the CDP WebSocket on 9222:

# Start the CDP server
curl -X POST http://localhost:3000/api/cdp/start

# Check status
curl http://localhost:3000/api/cdp/status

# Bind to all interfaces on a custom port
curl -X POST http://localhost:3000/api/cdp/settings \
  -H "Content-Type: application/json" \
  -d '{"host": "0.0.0.0", "port": 9333}'

CDP bootstrap endpoints (port 9222)

The CDP server speaks the Chrome-compatible HTTP bootstrap surface that every client library uses to discover WebSocket URLs from just a base URL:

WebSocket endpoints: ws://host:9222/devtools/browser/{uuid} (root connection, used by puppeteer.connect and connectOverCDP) and ws://host:9222/devtools/page/{targetId} (page-scoped attach, used by CRI-style clients).

First connection - pick your client

The same CDP endpoint behind three different idiomatic client APIs. All three spawn a new purple “CDP” tab inside LumaBrowser; the Lumabyte LLM-fallback domain is available from every one.

// npm install puppeteer-core
const puppeteer = require('puppeteer-core');

// LumaBrowser is already running; no bundled Chromium to launch.
const browser = await puppeteer.connect({
  browserURL: 'http://127.0.0.1:9222',
});

// Spawns a new "CDP" tab inside LumaBrowser.
const page = await browser.newPage();
await page.goto('https://example.com');

console.log(await page.title()); // "Example Domain"

await browser.disconnect(); // leaves the tab open; .close() would close it

// npm install playwright
const { chromium } = require('playwright');

// Attach to the existing LumaBrowser CDP endpoint. connectOverCDP is the
// documented path for driving a pre-existing browser instance.
const browser = await chromium.connectOverCDP('http://127.0.0.1:9222');

// LumaBrowser exposes an implicit default BrowserContext, so
// browser.contexts()[0] resolves immediately on connect.
const context = browser.contexts()[0] ?? await browser.newContext();
const page    = await context.newPage();
await page.goto('https://example.com');

console.log(await page.title()); // "Example Domain"

await browser.close();

// npm install chrome-remote-interface
const CDP = require('chrome-remote-interface');

// Raw CDP client - no higher-level page abstraction. Useful when you want
// to dispatch protocol commands directly, or build a thin wrapper.
const client = await CDP({ host: '127.0.0.1', port: 9222 });
const { Page, Runtime } = client;

await Page.enable();
await Page.navigate({ url: 'https://example.com' });
await Page.loadEventFired();

const { result } = await Runtime.evaluate({
  expression: 'document.title',
  returnByValue: true,
});
console.log(result.value); // "Example Domain"

await client.close();

Lumabyte CDP Domain - five methods

All five route through the same LLM-fallback orchestrator as the Selenium driver. Dispatch them from any CDP session, Puppeteer, Playwright, or raw CDP clients.

Lumabyte.find - resolve a selector, deterministically first

Accepts a description, a selector, or both. If both are given the server tries the selector against DOM.querySelector first and only invokes the LLM when it misses. Returns the selector that actually resolved, plus which strategy produced it.

const page = await browser.newPage();
await page.goto('https://example.com');

const cdp = await page.target().createCDPSession();
await cdp.send('Lumabyte.configureFallback', { enabled: true });

const r = await cdp.send('Lumabyte.find', {
  description: 'the more information link',
  selector:    'a.deprecated-class',  // optional hint tried first
});

console.log(r); // { selector: 'a[href*="iana.org"]', strategy: 'description' }

const context = browser.contexts()[0] ?? await browser.newContext();
const page    = await context.newPage();
await page.goto('https://example.com');

// newCDPSession is the canonical way to dispatch custom CDP commands
// from Playwright.
const cdp = await context.newCDPSession(page);
await cdp.send('Lumabyte.configureFallback', { enabled: true });

const r = await cdp.send('Lumabyte.find', {
  description: 'the more information link',
  selector:    'a.deprecated-class',  // optional hint tried first
});

console.log(r); // { selector: 'a[href*="iana.org"]', strategy: 'description' }

const CDP = require('chrome-remote-interface');

const client = await CDP({ host: '127.0.0.1', port: 9222 });
const { Page } = client;

await Page.enable();
await Page.navigate({ url: 'https://example.com' });
await Page.loadEventFired();

// Custom domains dispatch via client.send(method, params).
await client.send('Lumabyte.configureFallback', { enabled: true });

const r = await client.send('Lumabyte.find', {
  description: 'the more information link',
  selector:    'a.deprecated-class',
});

console.log(r); // { selector: 'a[href*="iana.org"]', strategy: 'description' }

await client.close();

Lumabyte.click - semantic click

Resolves via the same pipeline as Lumabyte.find, then scrolls the element into view with DOM.scrollIntoViewIfNeeded, reads its box model with DOM.getBoxModel, and dispatches a synthetic mouse press/release at the center via Input.dispatchMouseEvent. Returns the winning selector, the strategy that produced it, and the click coordinates.

const cdp = await page.target().createCDPSession();
await cdp.send('Lumabyte.configureFallback', { enabled: true });

const clicked = await cdp.send('Lumabyte.click', {
  description: 'the add to cart button',
  selector:    'button.buy-now',  // fast path; falls through if absent
});
// { clicked: true, selector: 'button.buy-now', strategy: 'selector', x: 412, y: 318 }

Lumabyte.domSnapshot - one round-trip page capture

Collapses DOMSnapshot.captureSnapshot plus optional Accessibility.getFullAXTree and Page.captureScreenshot into one CDP call. Useful for agent loops that want a full page state in a single round-trip instead of orchestrating three separate commands.

const cdp = await page.target().createCDPSession();

const { snapshot, axTree, screenshot } = await cdp.send('Lumabyte.domSnapshot', {
  includeAxTree:     true,
  includeScreenshot: true,
});

// snapshot   - DOMSnapshot.captureSnapshot result
// axTree     - Accessibility.getFullAXTree result (undefined if not requested)
// screenshot - base64 PNG string (undefined if not requested)

Lumabyte.configureFallback - per-session fallback config

CDP has no first-class capability slot (unlike WebDriver's lumabyte:llmFallback), so per-session LLM behavior is configured with this method. Accepts a boolean shortcut or a full config object. Settings persist for the lifetime of the CDP session.

const cdp = await page.target().createCDPSession();

// Full config form
await cdp.send('Lumabyte.configureFallback', {
  enabled:            true,
  onFindFail:         true,
  onClickIntercepted: true,
  slot:               null,  // null → use the default selector-resolver slot
});

// Boolean shortcut - equivalent to { enabled: true, onFindFail: true, onClickIntercepted: true }
await cdp.send('Lumabyte.configureFallback', true);

Lumabyte.getInfo - fallback capability probe

Reports whether an LLM fallback service is wired on the server side and the current session's effective config. Use it at the start of a session to decide whether Lumabyte.find/Lumabyte.click can safely be called with description-only arguments.

const cdp = await page.target().createCDPSession();
const info = await cdp.send('Lumabyte.getInfo');

// {
//   fallbackAvailable: true,
//   config:   { enabled: false, onFindFail: false, onClickIntercepted: false, slot: null },
//   defaults: { defaultEnabled: false, onFindFail: true, onClickIntercepted: true, slot: null }
// }

if (info.fallbackAvailable && !info.config.enabled) {
  await cdp.send('Lumabyte.configureFallback', true);
}

LLM Slot

All CDP LLM-fallback paths, Lumabyte.find, Lumabyte.click, and description-based retries triggered by configured fallback, route through the same shared core-scope selector-resolver slot as the Selenium driver. Configure a fast, low-latency model there once and every description-based call from either driver benefits. If the slot is unconfigured, the resolver falls back to your global active LLM.

End-to-end example

Mixing native CDP with Lumabyte.* in a realistic flow. Same LumaBrowser endpoint, three idiomatic client APIs.

const puppeteer = require('puppeteer-core');

const browser = await puppeteer.connect({ browserURL: 'http://127.0.0.1:9222' });
const page    = await browser.newPage();
await page.goto('https://example.shop/products/coffee-grinder');

const cdp = await page.target().createCDPSession();
await cdp.send('Lumabyte.configureFallback', { enabled: true });

// Semantic click - no brittle CSS needed.
await cdp.send('Lumabyte.click', { description: 'the add-to-cart button' });
await cdp.send('Lumabyte.click', { description: 'the cart icon in the header' });

// One round-trip: DOMSnapshot + AX tree + viewport screenshot.
const snap = await cdp.send('Lumabyte.domSnapshot', {
  includeAxTree:     true,
  includeScreenshot: true,
});

// Native CDP still works; verify via Runtime.evaluate.
const { result } = await cdp.send('Runtime.evaluate', {
  expression: 'document.title',
  returnByValue: true,
});
console.log(result.value);

await browser.disconnect();

const { chromium } = require('playwright');

const browser = await chromium.connectOverCDP('http://127.0.0.1:9222');
const context = browser.contexts()[0] ?? await browser.newContext();
const page    = await context.newPage();
await page.goto('https://example.shop/products/coffee-grinder');

const cdp = await context.newCDPSession(page);
await cdp.send('Lumabyte.configureFallback', { enabled: true });

// Semantic click - no brittle CSS needed.
await cdp.send('Lumabyte.click', { description: 'the add-to-cart button' });
await cdp.send('Lumabyte.click', { description: 'the cart icon in the header' });

// One round-trip: DOMSnapshot + AX tree + viewport screenshot.
const snap = await cdp.send('Lumabyte.domSnapshot', {
  includeAxTree:     true,
  includeScreenshot: true,
});

// Playwright's native locators still work; mix and match freely.
console.log(await page.title());

await browser.close();

const CDP = require('chrome-remote-interface');

const client = await CDP({ host: '127.0.0.1', port: 9222 });
const { Page, Runtime } = client;

await Page.enable();
await Page.navigate({ url: 'https://example.shop/products/coffee-grinder' });
await Page.loadEventFired();

await client.send('Lumabyte.configureFallback', { enabled: true });

// Semantic click - no brittle CSS needed.
await client.send('Lumabyte.click', { description: 'the add-to-cart button' });
await client.send('Lumabyte.click', { description: 'the cart icon in the header' });

// One round-trip: DOMSnapshot + AX tree + viewport screenshot.
const snap = await client.send('Lumabyte.domSnapshot', {
  includeAxTree:     true,
  includeScreenshot: true,
});

const { result } = await Runtime.evaluate({
  expression: 'document.title', returnByValue: true,
});
console.log(result.value);

await client.close();

Architecture notes for evaluators

• Only ~10 handwritten handlers. The server implements Browser.*, Target.*, and Lumabyte.* itself; every other command is forwarded through webContents.debugger.sendCommand. You get the ~290 commands Chromium implements natively for free, and the translation surface stays small and auditable.
• Session model. The server supports flat-mode sessions (Target.setAutoAttach with flatten: true), matching what modern Puppeteer and Playwright's connectOverCDP both expect. Multiple sessions can attach to the same target; events fan out to every subscriber.
• Browser contexts. An implicit default context is advertised on connect so Playwright's browser.contexts()[0] resolves. Target.createBrowserContext returns a new context ID that's reused as a persistent partition name (persist:cdp-ctx-<id>) when creating targets, so cookies and storage are isolated per context.
• Security posture. The CDP server binds to 127.0.0.1 by default and is gated by the extension's autostart setting. CDP itself has no authentication; do not expose it on an untrusted network.

Related

• Selenium vs Puppeteer vs Playwright - side-by-side comparison of all three drivers and when to pick each.
• LumaBrowser vs Puppeteer - focused deep-dive on the Puppeteer connection story.
• LumaBrowser vs Playwright - honest breakdown of what connectOverCDP does and doesn't give you.
• Selenium Driver - the WebDriver analog of this extension.
• LLM Fallback - shared resolution pipeline both drivers route through.

LumaBrowser is built on a modular extension architecture. Most major features (network watcher, AI chat, timed tasks, page change detector, the WebDriver and CDP drivers) are extensions that can be enabled, disabled, or replaced.

Built-in Extensions

LLM Providers

Each extension can use a different LLM independently via the slot-based routing system:

• Local LLM Server - bundled, managed llama.cpp server with one-click hardware-aware model setup (Gemma 4 / Qwen 3.6 GGUF lineup), no API key. See Local AI / LLM Server.
• OpenAI-compatible - LM Studio, Ollama, or any OpenAI API endpoint
• Anthropic - Claude API

Custom Extensions

LumaBrowser supports a plugin architecture for building custom extensions with their own UI, REST routes, and MCP tools. Extension development documentation will be available soon.

Free, no-auth-required helper APIs. These are hosted at https://lumabyte.com; they are not part of the LumaBrowser desktop app and will 404 on http://localhost:3000. Use them from any client, no install required. Subject to change, but we'll keep them running as long as possible.