Requests and provider adapters

Declare an endpoint once

An Endpoint describes a reusable operation. Binding produces a Request carrying the method, URL, headers and optional body. Both can be built without a client.

from common_http import Endpoint, HttpClient, ListStyle, Parameter, Request

search = Endpoint(
    name="quotes",
    url="/v1/{market}/quotes",
    params=(
        Parameter.path("market"),
        Parameter.query("symbols", alias="symbol", type="str", list_style=ListStyle.REPEAT),
        Parameter.query("limit", type="int", default=20, minimum=1, maximum=100),
    ),
)
http = HttpClient.create(base_url="https://provider.example/")
message = http.build_request(search, market="us", symbols=["SPY", "QQQ"])
assert message.url.query.get_all("symbol") == ("SPY", "QQQ")
standalone = search.request(base_url="https://provider.example/", market="us", symbols=["SPY"])
exact = Request.get("https://provider.example/health")

Enum values accept their strings: type="str", kind="query" and list_style="repeat" work. Types include strings, integers, floats, booleans, dates and datetimes. Incompatible inputs fail validation. Lists declare repeated, comma, space or pipe serialization. Parameters support aliases, required/default values, choices, bounds, item counts and date/time formats. Path values are encoded as individual segments. Headers and cookies have Parameter.header and Parameter.cookie constructors.

Unknown endpoint values fail unless allow_extra=True. required_any declares groups requiring at least one supplied parameter. Supplied None removes an optional value; omission permits its default. An empty string remains a value. Required parameters cannot be removed. For names colliding with client arguments:

endpoint = Endpoint(name="search", url="/search", params=(Parameter.query("options"),))
message = http.build_request(endpoint, values={"options": "compact"})

Defaults and exact messages

build_request combines client header/query defaults before caller values. Endpoint declarations control binding and removal. request builds then sends; send does not reapply header/query defaults to a resolved Request. Configured cookie and authentication policies still apply. Provider enums, credential stores and domain completeness metadata belong to consumers, outside these contracts.

URLs and repeated fields

For base_url="https://host.example/api/", "items" resolves under /api/ and "/items" resolves at the host root. Keep the trailing slash for a directory base. Absolute targets also work. Fragments are not sent in the HTTP request target.

from common_http import Headers, QueryParams, Request, URL

query = QueryParams.from_pairs([("symbol", "SPY"), ("symbol", "QQQ"), ("flag", None)])
url = URL.parse("https://provider.example/search").with_query(query)
headers = Headers.from_pairs([("Accept", "application/json"), ("X-Tag", "a"), ("X-Tag", "b")])
message = Request.get(url, headers=headers)
assert headers.get_all("x-tag") == ("a", "b")

QueryParams(encoded=...) preserves wire spelling. Ordered pairs retain repetition; low-level pair None means a bare key. Native request params={"key": None} instead removes that key. Header lookup is case insensitive; get_all retains all occurrences. append, replace and without return new values.

Body formats

Supply one body input. Typed Body values and shortcuts share validation.

from common_http import Body, BodyPart, Request

url = "https://provider.example/upload"
messages = [
    Request.post(url, json={"symbols": ["SPY"]}),
    Request.post(url, form=[("symbol", "SPY"), ("symbol", "QQQ")]),
    Request.post(url, text="hello"),
    Request.post(url, binary=b"\x00\x01"),
    Request.post(
        url,
        body=Body.multipart(
            [
                BodyPart(name="note", content="quarterly"),
                BodyPart(
                    name="file",
                    filename="data.csv",
                    content=b"symbol\nSPY\n",
                    content_type="text/csv",
                ),
            ]
        ),
    ),
]

json=None sends JSON null; it differs from no body. Form fields are text pairs. Multipart parts may repeat names and carry headers. The transport generates boundaries and framing. Body.binary(data, content_type="application/pdf") sets a binary media type. Live upload iterators and file-handle streaming are outside the current body contracts.

Site adapters and embedded chart data

Inspect returned HTML first. A chart can live in attributes or scripts even when the page displays it using JavaScript. Verify that behavior per site.

import json
from html.parser import HTMLParser

from common_http import Endpoint, HttpClient


class FlowParser(HTMLParser):
    def __init__(self):
        super().__init__()
        self.rows = None

    def handle_starttag(self, tag, attrs):
        fields = dict(attrs)
        if fields.get("id") == "fund-flow-chart-container" and fields.get("data-series"):
            self.rows = json.loads(fields["data-series"])


fund_page = Endpoint(name="fund_page", url="/etf/{symbol}/")


async def fetch_flows(http: HttpClient, symbol: str):
    result = await http.request(fund_page, symbol=symbol, options={"decode": "text"})
    parser = FlowParser()
    parser.feed(result.unwrap().payload)
    if parser.rows is None:
        raise ValueError("Expected fund-flow data is absent from the returned HTML")
    return parser.rows

The consumer supplies the ETFdb base URL and owns markup, units, dates and data completeness. The notebook verifies this path with fixtures. Authentication walls, rate limits and markup changes remain visible failures. Browser fallback is a separate choice; this HTTP package does not require a browser installation.