Skip to content

common_db.client.converters

converters.py — any format in, any format out, Arrow as the lossless spine.

IN (to_arrow): polars · pandas · arrow · list[dict] · list[pydantic rows] · dict-of-lists · csv/parquet paths OUT (from_arrow): polars · pandas · arrow · rows (tuples) · dicts — with index and per-column dtype control (ReadConfig).

Pydantic is one dialect among equals (Sam’s correction). pandas is a GUEST: converted faithfully when present, never imported unless the caller brought pandas data or asked for pandas out.

Functions

from_arrow

def from_arrow(table: pa.Table, output: OutputKind | str = OutputKind.POLARS, *, index: tuple[str, ...] = (), dtypes: dict[str, str] | None = None) -> Any

Arrow -> the requested container, with index/dtype control.

dtypes uses arrow type names (‘int64’, ‘float64’, ‘string’, …) — applied by cast BEFORE conversion so every output format sees them.

iter_copy_rows

def iter_copy_rows(table: pa.Table, *, batch_rows: int = 50000) -> Any

Yield python-native row tuples (psycopg’s copy().write_row food) — memory-streaming, zero disk, zero csv round-trip (the legacy killer). batch_rows is the flush granularity (WriteConfig.batch_rows, wired).

to_arrow

def to_arrow(data: Any) -> pa.Table

Any supported source -> pa.Table (column order preserved, types kept).