Self-Healing Playwright via connectOverCDP with LLM Selector Fallback

Playwright is a cross-browser automation library that can drive its own protocol natively or attach to an existing Chromium over CDP. LumaBrowser is the browser itself, exposing CDP on port 9222 and the Lumabyte.* domain for LLM-resolved selectors. chromium.connectOverCDP is the documented path — the same path every cloud-browser provider (Browserless, Browserbase, Steel.dev) uses — so the move is a one-line change.

LumaBrowser’s cdp-driver extension is a standards-compliant CDP WebSocket server on port 9222 — the same port and discovery surface Chromium exposes when launched with --remote-debugging-port=9222. Playwright’s chromium.connectOverCDP is the documented path for driving a pre-existing Chromium over CDP. What you don’t have to change:

• No new library. Keep your existing playwright or @playwright/test install. Works from JavaScript/TypeScript, Python, .NET, and Java bindings — they all speak connectOverCDP.
• Locators, auto-wait, and the Page API are unchanged. Those are client-side features. The driver just sees the resulting CDP commands, which forward through to Chromium as normal.
• browser.contexts()[0] resolves on connect. LumaBrowser advertises a default BrowserContext so Playwright’s “attach to the existing context” idiom works immediately.
• Native CDP is untouched. Network.setUserAgentOverride, Fetch.enable, Emulation.setDeviceMetricsOverride, DOMSnapshot.captureSnapshot — all pass through to Chromium.
• page.context().newCDPSession(page) still works. That’s how you dispatch custom CDP domains from Playwright — including LumaBrowser’s Lumabyte.* methods.

Just change the connect:

// Launches a bundled Chromium binary.
const { chromium } = require('playwright');

const browser = await chromium.launch({
  headless: false,
});
const context = await browser.newContext();
const page    = await context.newPage();

await page.goto('https://example.com');
console.log(await page.title());

await browser.close();

// LumaBrowser is already running. Attach.
const { chromium } = require('playwright');

const browser = await chromium.connectOverCDP(
  'http://127.0.0.1:9222',
);
const context = browser.contexts()[0]
  ?? await browser.newContext();
const page    = await context.newPage();

await page.goto('https://example.com');
console.log(await page.title());

await browser.close();

That’s the entire migration. Change launch(...) to connectOverCDP(url), pick up the implicit default context, and point at port 9222. Everything below is what you gain by opting in.

Three concrete wins. Each one is opt-in: pay the cost only where you want the benefit.

Playwright’s locators are already smart (role-based, text-based, test-id-first) — but they still resolve against a concrete DOM. Change a data-testid and getByTestId('submit') breaks the same way any other selector does. LumaBrowser’s Lumabyte.* CDP domain adds a description-based resolver that retries via an LLM using a natural-language description of what the element is, not how it’s currently labeled.

You dispatch it via page.context().newCDPSession(page) — Playwright’s canonical escape hatch for raw CDP.

// Breaks the moment the test-id is renamed
// during a UI refresh.
const { chromium } = require('playwright');

const browser = await chromium.launch();
const page = await (await browser.newContext()).newPage();
await page.goto('https://example.shop/cart');

await page.getByTestId('signup-btn').click();

// LLM fallback re-resolves when selectors miss.
const { chromium } = require('playwright');

const browser = await chromium.connectOverCDP(
  'http://127.0.0.1:9222',
);
const context = browser.contexts()[0]
  ?? await browser.newContext();
const page = await context.newPage();
await page.goto('https://example.shop/cart');

const cdp = await context.newCDPSession(page);
await cdp.send('Lumabyte.configureFallback', {
  enabled: true,
});

// Option A: keep your locator, add a description fallback.
await cdp.send('Lumabyte.click', {
  description: 'the sign-up button',
  selector:    '[data-testid="signup-btn"]',
});

// Option B: skip selectors entirely.
await cdp.send('Lumabyte.click', {
  description: 'the sign-up button',
});

Playwright agent loops that need full page state typically make three separate round-trips: DOMSnapshot.captureSnapshot, Accessibility.getFullAXTree, and Page.captureScreenshot. LumaBrowser’s Lumabyte.domSnapshot returns all three in a single call, ordered and correlated server-side — and it does this over the same connectOverCDP session you already have.

This matters especially for Playwright via connectOverCDP: Playwright adds a Node.js driver hop between your client and the browser for every protocol call. Cutting 3 × RTT to 1 × RTT is meaningful when you’re paying two network hops per round-trip.

// Three CDP round-trips, each going client →
// Playwright driver → browser → back.
const cdp = await page.context().newCDPSession(page);
await cdp.send('DOM.enable');
await cdp.send('Accessibility.enable');

const snapshot = await cdp.send(
  'DOMSnapshot.captureSnapshot',
  { computedStyles: [] }
);
const axTree = await cdp.send(
  'Accessibility.getFullAXTree'
);
const { data: screenshot } = await cdp.send(
  'Page.captureScreenshot',
  { format: 'png' }
);

// One call, server-correlated payload.
const cdp = await page.context().newCDPSession(page);

const { snapshot, axTree, screenshot } =
  await cdp.send('Lumabyte.domSnapshot', {
    includeAxTree:     true,
    includeScreenshot: true,
  });

// snapshot   — DOMSnapshot.captureSnapshot
// axTree     — Accessibility.getFullAXTree
// screenshot — base64 PNG of the viewport
// 1 request. Parse locally.

Classic Playwright ships a ~170 MB Chromium per-platform binary, downloads it during playwright install, and launches it as a child process for every script run. CI caches drift, --user-data-dir trees pile up, the binary version lags Chrome stable. LumaBrowser’s CDP server runs inside the browser process itself — no binary download, no launcher lifecycle, no temp profile to reap.

And unlike attaching to a user’s regular Chrome (where your automation would fight for the same tabs the user is browsing in), LumaBrowser gives the client dedicated automation tabs: only tabs created with kind: 'cdp' surface as CDP targets, rendered in the tab strip with a purple accent and a CDP badge so you can watch the session drive them live.

// playwright install downloads a bundled
// Chromium and spawns it per run.
const { chromium } = require('playwright');

const browser = await chromium.launch({
  headless: false,
  args: ['--no-sandbox'],
});

// Plus: Chromium version drift, ~170MB
// per-platform binary download during
// `playwright install`, stale --user-data-dir
// trees, and an occasional "browser closed
// unexpectedly" from the launcher at 3am.

// LumaBrowser is already running. Attach.
const { chromium } = require('playwright');

const browser = await chromium.connectOverCDP(
  'http://127.0.0.1:9222',
);

// Dedicated "CDP" tabs, visible in the strip.
// No binary download — skip `playwright install`
// for the chromium target.
// No child process.
// No --user-data-dir to clean up.
// The user's regular tabs stay invisible to
// the CDP client, so Network Interceptor and
// AI Chat don't fight for the debugger lock.

What a realistic flow looks like when you mix Playwright locators, native CDP, and Lumabyte.*.

const { chromium } = require('playwright');

// 1. Attach to the already-running browser.
const browser = await chromium.connectOverCDP('http://127.0.0.1:9222');
const context = browser.contexts()[0] ?? await browser.newContext();
const page    = await context.newPage();

await page.goto('https://example.shop/products/coffee-grinder');

const cdp = await context.newCDPSession(page);

// 2. Opt in to LLM fallback for this session.
await cdp.send('Lumabyte.configureFallback', {
  enabled:            true,
  onFindFail:         true,
  onClickIntercepted: true,
});

// 3. Mix Playwright's native locators with Lumabyte.click for anything
//    where the DOM label is less stable than the intent.
await page.getByRole('button', { name: /sign up/i }).click();  // locator
await cdp.send('Lumabyte.click', { description: 'the add-to-cart button' });
await cdp.send('Lumabyte.click', { description: 'the cart icon in the header' });

// 4. Pull DOM + AX tree + screenshot in one round-trip.
const { snapshot, axTree, screenshot } = await cdp.send('Lumabyte.domSnapshot', {
  includeAxTree:     true,
  includeScreenshot: true,
});

// 5. Playwright's own API still works on top of the same page.
console.log(await page.title());

await browser.close();

Playwright’s locator API for the elements that are stable, description-based resolution for the ones that aren’t, and one round-trip for full page capture when the agent needs it. Same page, same session, no context switching between tools.

@playwright/mcp is Microsoft’s MCP server for Playwright — the idiomatic way to give an LLM agent (Claude Desktop, Cursor, OpenClaw, any MCP-compatible host) tool-driven control of a browser. It supports --cdp-endpoint for attaching to an existing browser, which is exactly what LumaBrowser exposes.

# Start LumaBrowser's CDP server (once per launch).
curl -X POST http://localhost:3000/api/cdp/start

# Point Playwright MCP at it. Your LLM agent now drives LumaBrowser.
npx @playwright/mcp --cdp-endpoint http://127.0.0.1:9222

The win: the MCP server’s tool calls land in a browser with built-in LLM selector fallback. When the agent produces a locator that doesn’t quite match, the Lumabyte orchestrator catches it instead of the agent loop crashing. And the agent’s own LLM doesn’t need to regenerate the tool call from scratch.

The full row-by-row matrix — connectOverCDP bootstrap, browser contexts, locators / auto-wait / expect, navigation and input, page.route, screenshots, Playwright MCP via --cdp-endpoint, the additive Lumabyte.* methods, plus the documented limits (Firefox / WebKit, full tracing, codegen, launch()) — lives in the API reference: /apis → CDP Driver section.

Two ways, takes under a minute either way:

1. From the UI

1. Open Settings → CDP Driver.
2. (Optional) Check “Start automatically when LumaBrowser launches.”
3. Click Save & Start.
4. Point your Playwright client at http://127.0.0.1:9222.

Or start it programmatically: POST /api/cdp/start. Full CDP domain reference, Lumabyte.* method signatures, host/port settings, and MCP tool surface are in the CDP Driver section of the API docs.

Install LumaBrowser, enable the cdp-driver extension, and swap chromium.launch() for chromium.connectOverCDP('http://127.0.0.1:9222'). Every Playwright locator, assertion, and Page API keeps working. The Lumabyte.* domain is there when you want it.