Browser
common-browser supplies the browser mechanics shared by applications, platform
adapters and agent tools. Playwright is the SDK implementation. Site selectors,
login steps, business records, mutation verification and agent reasoning stay in
their consumers.
Install and start
uv add 'common-browser[playwright]'uv run playwright install chromiumPython 3.12 or newer is required. The base distribution imports without the optional SDK; opening a browser requires the extra and suitable browser binaries. Run async examples in an async function or a notebook with top-level await.
from common_browser import Browser, Target
async with Browser.create(headless=True) as browser: context = await browser.new_context() page = await context.new_page() (await page.goto("https://example.com/")).unwrap() heading = (await page.read(scope=Target.css("h1"))).unwrap().contentOne outer context closes the children it creates. Optional context/page contexts
release children earlier. Resource acquisition returns handles and raises
BrowserError on failure. Browser operations return BrowserResult[T].
Entry points and ownership
| Need | Surface |
|---|---|
| Supplied configuration fields | Browser.create(**settings) |
| Validated application configuration | Browser.from_config(BrowserConfig(...)) |
| Borrow an async Playwright browser/context/page | Browser.attach(native_handle) |
| Inspect declared capabilities and lifecycle | browser.info |
| Create an isolated context | await browser.new_context(ContextConfig(...)) |
| Select existing contexts/pages | browser.contexts, context.pages, context.select_page(ref) |
| Keep related account operations together | async with context.sequence(): |
| Use advanced SDK operations | async with page.native() as native_page: |
| Contain a complete workflow for a sync caller | SyncBrowser.run(workflow, config=...) |
Construction is unconnected. open() or entering the async context acquires
resources. Each runtime and supplied handle belongs to its owning loop/thread.
Borrowed parents remain alive after wrapper cleanup; children created through
the wrapper are owned. Cancel or drain active work before closing its resource.
The wrapper infers a borrowed handle’s engine when the SDK exposes its browser.
A context without that metadata requires an explicit engine in its borrowed
BrowserConfig; an explicit mismatch is rejected.
Context and Page are handles into one shared pipeline. BaseBrowser owns
admission, locks, deadlines and reports; PlaywrightBrowser implements SDK seams.
Actions and reading are aspects of this same backend axis, with shared targeting.
There is no second retry engine or per-site browser class.
Connection modes and configuration
| Mode | Construction | Context behavior |
|---|---|---|
launch | Browser.create(engine="chromium", headless=True) | Create isolated contexts; Firefox/WebKit are also configurable |
persistent | Browser.create(mode="persistent", profile_dir=profile_path) | One browser.default_context; profile access is exclusive |
remote | Browser.create(mode="remote", endpoint=websocket_url) | Native Playwright connection; create/select contexts explicitly |
cdp | Browser.create(mode="cdp", endpoint=cdp_url) | Chromium; select existing contexts explicitly |
| Borrowed | Browser.attach(native_handle) | Supplied context/page has an explicit default context; supplied browser contexts are selected explicitly |
Native remote connections require matching Playwright major/minor versions. CDP supports Chromium and has different capabilities from the native protocol. The engine container’s deployment, network access and tenant/profile allocation belong to the application. The client still installs the SDK. Playwright connection reference.
BrowserConfig controls engine/mode, endpoint/connection headers, local launch
options/proxy/profile, startup timeout, operation timeout and artifact/reference
bounds. ContextConfig controls viewport, locale/timezone, user agent, permissions,
downloads, TLS behavior, JavaScript, service workers, HTTP credentials/headers,
initial storage and optional allowed origins. model_json_schema() exposes the
configuration vocabulary. Contradictory mode settings fail at construction.
An omitted engine (None) uses Chromium for launch/remote connection and infers
the engine from a borrowed handle when possible; an explicit engine must match,
and JSON round trips preserve that distinction.
OperationConfig(timeout=...) uses seconds and can take an absolute monotonic
deadline. It has no retries. context.sequence() is task-owned and reentrant;
sibling tasks do not inherit its permission to bypass the gate. Independent
contexts can run concurrently. Coordination covers handles registered in this
runtime, not independent external CDP clients.
Actions, targets and expectations
Target.css, .role, .label, .text, .placeholder, .test_id, .title and
.alt describe queries. .inside(...), .in_frame(...) and .nth(...) add
explicit scope or disambiguation. Ambiguous action targets fail rather than
silently selecting an element. Open shadow DOM follows SDK locator behavior.
Text matching also accepts TextPattern(pattern=..., flags=...) or a compiled
re.Pattern[str], for example Target.role("button", name=re.compile("accept|confirm", re.I)).
Pattern and flags have an explicit JSON object representation; ordinary strings
retain their existing meaning. Flags are limited to IGNORECASE, MULTILINE,
DOTALL and UNICODE. Regex syntax follows Playwright/JavaScript; Python-only
constructs are not translated. CSS, role kinds and test IDs remain plain strings.
Page conveniences include navigation/history, click/double-click, fill/type,
press, hover, check, select, scroll, drag, upload and element-state waits.
page.wait(target, state="visible") also accepts hidden, attached and
detached. These conveniences normalize through the same Action command path
as perform and share the operation deadline.
page.wait_for_load(state="load") also accepts domcontentloaded and
networkidle; deciding which state establishes application readiness stays with
the caller. page.is_enabled(target) reports the SDK’s effective enabled state,
including disabled fieldsets and ARIA state, through BrowserResult[bool].
page.select(target, values=...) selects option values; use labels=... or
indices=... for those identities instead. Supply exactly one axis; an empty
tuple clears selection. page.upload(target, files) accepts either consumer-local
file paths or UploadFile(name=..., mime_type=..., content=SecretBytes(...))
inputs. In-memory payloads are masked in repr/serialization and limited to 50 MiB
in total by the SDK. Mixed path/payload sequences are rejected; an empty tuple
clears the file input.
from common_browser import Action, EventWait, Target
async with context.sequence(): (await page.fill(Target.label("Subject"), subject)).unwrap() report = await page.perform( Action.click(Target.role("button", name="Export")), expect=EventWait.download(to=export_path), ) artifact = report.unwrap().artifactExpectations are armed before dispatch. perform returns
BrowserResult[EventData | None]; event kind distinguishes request, response,
navigation, popup, download, dialog and console facts. Popup events carry a
PageRef for context selection. Downloads become artifacts after transfer.
Dialog acceptance is explicit through EventWait.dialog(accept=True).
For a callback or ongoing observation, use a bounded subscription:
from common_browser import EventWait, OperationConfig
async with context.watch(EventWait.request(callback_pattern)) as callbacks: await provider_login_sequence(page) observed = (await callbacks.next(options=OperationConfig(timeout=10))).unwrap() callback_url = observed.urlRequest evidence can survive a failed callback navigation. Queue overflow is reported as partial evidence; listeners close with their subscription/context. The application decides which callback is authoritative.
Request/response events omit headers and bodies by default. Opt in with
EventWait.response(..., header_names=("content-type",), include_body=True, max_bytes=65536) or the same arguments on EventWait.request. Selected header
names are normalized to lowercase. EventData.headers contains SecretStr
values and .body contains optional SecretBytes; .get_secret_value() is the
explicit reveal boundary. max_bytes bounds retained URL/method/text, selected
headers and body together; oversized capture is rejected while available event
evidence remains accessible. The SDK may buffer data before this check. The
default limit is 32 MiB and the default watch queue capacity is 128 events;
EventWait(kind=..., capacity=...) configures the queue bound.
Read, scan and extract
| Verb | Data inside the result | Meaning |
|---|---|---|
read | PageContent | Bounded text/HTML for a page or scoped target |
scan | PageSnapshot | Bounded DOM elements, forms, links and optional frame/accessibility observations |
extract / extract_with | Extraction[T] | Caller-typed items with aligned document/row provenance |
evaluate | SDK-serialized value or caller-decoded T | Explicit JavaScript evaluation; can have side effects |
ElementRef identifies the node observed by scan, unlike a query that can
resolve again. Document replacement, node replacement/detachment, identifying
changes, eviction and native access invalidate affected observations. The
caller rescans after a stale result. Business-record identity still needs the
consumer’s own checks.
from pydantic import BaseModelfrom common_browser import FieldRead, Target
class Product(BaseModel): name: str price_text: str
report = await page.extract( Product, rows=Target.css(".product"), fields={ "name": FieldRead(target=Target.css(".name")), "price_text": FieldRead(target=Target.css(".price")), }, limit=100,)if report.ok: products = report.unwrap().itemselse: admitted = report.value.items if report.value is not None else () issues = report.issuesField reads support text, HTML, attributes and scalar form values. Optional
missing fields are omitted so model defaults can apply. Required, ambiguous,
invalid or oversized values produce indexed issues; oversized fields are not
truncated into valid-looking records. extract_with(decoder, ...) accepts a
typed conversion callable separately from declarative configuration.
Synchronous caller decoders run cooperatively and cannot be preempted while
executing. If decoding exceeds the operation deadline, the final report is a
timeout and retains the data already decoded.
ReadConfig, ScanConfig and ExtractConfig provide the equivalent validated
input objects. Coverage belongs to the result; observations and extracted items
remain data. Per-row scalar capture does not establish whole-list atomicity,
complete pagination or complete virtualized data. Automatic scans avoid form
input values; explicit field/evaluation access remains the caller’s decision.
page.evaluate(expression, arg=None, decoder=..., options=...) optionally applies
a synchronous typed decoder to the SDK value. For example:
count = ( await page.evaluate( "() => document.querySelectorAll('.product').length", decoder=int, )).unwrap()Decoder failures produce a rejected result with the observed SDK value retained
in .evidence. Awaitable decoder results are rejected. These decoders share the
cooperative deadline boundary above; JavaScript evaluation still executes only
once and invalidates observed references afterward.
Results and failures
Use .ok, .status, .reason, .phase, .issues and .coverage for handling.
.value and .evidence retain explicitly accessible products; routine repr
omits payloads and secrets. unwrap() rejects errors and partial outcomes.
Phase not_started, invoked or completed describes SDK progress. Completed
does not prove a business mutation. A timeout after invocation can leave an
uncertain application outcome. The browser never automatically repeats the
action; provider adapters own submit-once decisions and independent rereads.
Cancellation propagates after cleanup and may retain a receipt as last_result.
Cleanup failures attach the same safe receipt to BrowserError.result and
browser.last_result; its original .error remains explicitly accessible but
hidden from repr. Drain active work before retrying aclose(). Cleanup can still
time out; a retained receipt does not imply closure succeeded.
Cookies, state, files and native access
context.cookies(), set_cookies(...) and clear_cookies(...) use the browser
jar. Cookie preserves name/domain/path/partition scope and wraps its value.
The URL construction convenience derives browser domain/path/secure scope;
explicit domain/path construction preserves chosen scope.
(await context.export_state()).unwrap() returns protected StorageSnapshot.
Supply it as ContextConfig(storage=snapshot) when creating a fresh context.
Its version, engine, SDK version, origins and feature names describe the snapshot.
Ordinary serialization masks the payload; .export() deliberately reveals a
copy for authorized storage. IndexedDB capture is explicit. Storage snapshots
do not include sessionStorage or constitute a full profile backup or proof of
current authentication.
Playwright authentication state.
Screenshots, downloaded files, traces and context.fetch(...) return
ArtifactData. Fetch uses the context’s authenticated request client; ordinary
HTTP belongs in common-http. Artifact byte limits bound admitted outputs; the
SDK can buffer a response before that limit is checked. Local paths always mean
the consumer filesystem. Remote downloads are transferred before becoming
usable local artifacts. Inject ArtifactSink for existing application storage.
Playwright download transfer.
page.screenshot(target=Target | ElementRef, to=...) captures one element.
Element capture rejects full_page=True, ambiguous queries and stale observed
references. Omitting the target retains page/full-page capture.
Native scopes provide the SDK escape hatch under exclusive context access. They invalidate affected observations afterward. The caller must adopt or close native-created children explicitly. Allowed-origin checks are navigation policy, not a network firewall; native scripts and borrowed handles cannot promise stronger enforcement than their actual boundary permits.
Compose auth, HTTP and synchronous consumers
Common-auth resolves supplied credentials, generates TOTP, validates OAuth caller/state/issuer/PKCE and exchanges codes through common-http. A provider adapter owns login selectors and consent steps. Browser state is not an auth token store, and cookie jars are not synchronized implicitly with common-http.
Existing consumers need an explicit compatibility adapter, not just changed imports. The grounded replacement map is:
| Consumer | Shared mechanics | Responsibilities retained by the consumer |
|---|---|---|
| Agents-core browser tools | Runtime/context/page ownership, targets, actions, reading, typed extraction and screenshots | Tool schemas, agent state/context, approvals, evidence policy and model interpretation |
| NC-feeds Schwab login | Browser UI steps and observed callback; common-auth validates/exchanges and common-http sends | Schwab selectors/stage decisions, Chrome/container deployment, keeper scheduling and recovery |
| Solutions platform BrowserPort | Navigation, regex actions, element/load waits, enabled checks, protected uploads, evaluation and authenticated artifacts | Locator fallback order, workspace/record identity, provider normalization, preflight, submit-once policy and independent verification |
Replacement simulations run the unchanged AppFolio and Meld clients with the
actual backend target resolver against synthetic local portals. AppFolio covers
acceptance, independent verification and authenticated artifacts; Meld covers
communication, audience selection, idempotent submission and a fresh
correspondence read. Both passed through the compatibility adapter. These
scenarios do not establish every provider action or live-site reliability.
Legacy synchronous url properties require a maintained observation/cache or a
deliberate interface change; they are not an instantaneous async page
observation. Advanced SDK-only behavior remains an explicit native scope.
| Common surface | Composition boundary |
|---|---|
common-auth + common-http | A bearer source plugs into HttpClient(auth=...); an observed browser callback feeds OAuthLogin.complete. Non-expiring FMP/EODHD keys use a supplied CredentialSource/API-key authenticator, not RemoteTokens. |
common-execution | Own workflow batching, retry and scheduling once above browser operations; browser actions are never automatically replayed. |
common-core | Reuse shared primitives where they fit; browser resource references remain scoped browser identities. |
common-rates | Reserved for shared quota coordination; this package does not implement or claim that integration. |
| Existing settings/Doppler and storage | Supply config/secrets through the existing settings boundary. Legacy nc-feeds request/result shapes and PostgreSQL token rows require explicit mapping; provider endpoints/parsing and database migrations stay outside browser. |
SyncBrowser.run(workflow, config=...) creates one event loop for the entire
owned async workflow and closes resources before returning detached data. It
rejects a caller thread that already has a running event loop and rejects
borrowed handles/live-resource results. Return supported scalar data, mappings,
standard containers, dataclasses or Pydantic models whose contents are also
detached. Protected cookie/storage products retain their secret wrappers.
Opaque custom objects, callbacks, generators and unawaited coroutines are
rejected. Async callers use the async core; an existing synchronous integration
can run its complete workflow in a worker thread. SDK objects remain
loop/thread-bound.
Playwright threading guidance.
Examples, verification and API
The usage notebook uses a local document and the public API. Class-based tests cover resource, action and observation behavior. Local composition tests exercise platform preflight/submit/reread, OAuth/TOTP with real common-auth/common-http, and an agent-style scan/action/read workflow. They also restore localStorage/IndexedDB into a new context and check authenticated fetches, redirects and output limits. These fixtures do not establish live provider login, bot-challenge behavior or every engine/mode combination.