common_dagster.specs
specs.py — the declaration vocabulary: @step · @sensor · @check · Job.
Everything here is a RECORD, not machinery: decorators attach specs to methods;
Job declares a cropping. Resolution into real Dagster objects happens in
build.py — declared data, constructed machinery (the house law).
Classes
CheckSpec
A custom check method bound to one product written by a step.
__init__
def __init__(fn: Callable, name: str, step: str, product: str | None = None, blocking: bool = False) -> NoneFields and properties declared here; inherited members belong to the bases above.
blocking
blocking: bool = Falsefn
fn: Callablename
name: strproduct
product: str | None = Nonestep
step: strJob
A cropping of the flow’s steps, runnable on its own terms.
config carries ONLY the row selector (e.g. {"run": "weekly"}) —
it travels as a run TAG (dg/run), never as baked per-op config, so
subset re-execution stays clean (audit: the old H1 rewind killer).
Knob VALUES live in the configs DB row, read live inside the step.
__init__
def __init__(steps: tuple[str, ...] | list[str], config: dict[str, str] | None = None, schedule: str | None = None, timezone: str = 'UTC', manual: bool = False, executor: str = 'multiprocess', resources: dict | None = None, max_concurrent: int | None = None, tags: dict[str, str] | None = None, job_type: JobType | None = None) -> NoneFields and properties declared here; inherited members belong to the bases above.
config
config: dict[str, str] | None = Noneexecutor
executor: str = 'multiprocess'job_type
job_type: JobType | None = Nonemanual
manual: bool = Falsemax_concurrent
max_concurrent: int | None = Noneresources
resources: dict | None = Noneschedule
schedule: str | None = Nonesteps
steps: tuple[str, ...] | list[str]tags
tags: dict[str, str] | None = Nonetimezone
timezone: str = 'UTC'SensorSpec
A real sensor method — arbitrary logic, resolved to a Dagster sensor.
requires upgrades the method to a HYBRID: the sensor monitors those asset
keys AND runs the body — the body receives upstream_done: bool as its
third parameter and decides (probe AND completion). Cursors advance only
when the body returns a RunRequest, so an unfired probe never eats the
upstream materialization.
__init__
def __init__(fn: Callable, name: str, job: str, interval: int = 30, requires: tuple[str, ...] = ()) -> NoneFields and properties declared here; inherited members belong to the bases above.
after
Read-only property.
after: tuple[str, ...]Compatibility spelling for declarations written before requires.
fn
fn: Callableinterval
interval: int = 30job
job: strname
name: strrequires
requires: tuple[str, ...] = ()SkipAsset
Bases: Exception
A deliberate successful no-output outcome for an optional operation.
Raise this from a step when there is no product to confirm. The compiler catches it at the Dagster boundary, emits no optional output, and therefore lets Dagster skip every dependent product confirmation honestly.
StepSpec
One operation and the persisted products it confirms. Captured by @step.
The operation always keeps name as its own Dagster node/asset identity.
produces is a separate, canonical product identity: the compiler creates
one confirmation asset per product and lets any number of operations write
it without creating duplicate Dagster definitions.
__init__
def __init__(fn: Callable, name: str, produces: tuple[str, ...] = (), deps: tuple[Any, ...] = (), group: str | None = None, checks: tuple[Any, ...] | dict[str, tuple[Any, ...]] = (), retry: RetryPolicy | None = None, isolate: bool = False, resources: dict | None = None, image: str | None = None, description: str | None = None, tags: dict[str, str] | None = None) -> NoneFields and properties declared here; inherited members belong to the bases above.
checks
checks: tuple[Any, ...] | dict[str, tuple[Any, ...]] = ()deps
deps: tuple[Any, ...] = ()description
description: str | None = Nonefn
fn: Callablegroup
group: str | None = Noneimage
image: str | None = Noneisolate
isolate: bool = Falsekey_name
Read-only property.
key_name: strBackward-compatible operation key leaf (always the descriptive name).
name
name: strparam_deps
Read-only property.
param_deps: tuple[str, ...]pytest-style data deps: method params beyond the reserved set.
produces
produces: tuple[str, ...] = ()product_names
Read-only property.
product_names: tuple[str, ...]Canonical persisted products confirmed after this operation.
products
Read-only property.
products: tuple[str, ...]Preferred spelling for the normalized persisted-product tuple.
resources
resources: dict | None = Noneretry
retry: RetryPolicy | None = Nonetags
tags: dict[str, str] | None = NoneFunctions
check
def check(*, step: str, product: str | None = None, blocking: bool = False, name: str | None = None) -> CallableDeclare a custom check for one product written by step.
product is optional only when the step declares exactly one product.
Requiring it for multi-product operations prevents a check from silently
attaching to the wrong persisted table.
sensor
def sensor(*, job: str, interval: int = 30, requires: tuple | list = (), after: tuple | list = (), name: str | None = None) -> CallableDeclare a real sensor method — probes, windows, cursors, Skip/Run.
requires=["container/asset", ...] makes it a HYBRID sensor: the body’s
signature becomes (self, context, upstream_done) and fires only on its
own verdict — the daily_ready pattern (vendor-served AND islands-done).
after= remains a compatibility alias and cannot be mixed with it.
step
def step(fn: Callable | None = None, *, produces: str | tuple[str, ...] | list[str] | None = None, deps: tuple | list = (), group: str | None = None, checks: tuple | list | dict[str, tuple | list] = (), retry: RetryPolicy | None = None, isolate: bool = False, resources: dict | None = None, image: str | None = None, description: str | None = None, tags: dict | None = None, name: str | None = None) -> CallableDeclare a flow method as a step (one asset, one node).
Laws enforced at class creation (DGFlow.init_subclass):
resources/image without isolate=True refuse loudly (audit rule 5 — they’d
be silently inert under the multiprocess executor).
Isolate pods run on the config-DB row’s defaults: launchpad overrides
beyond run never reach them (only the dg/run tag crosses the pod).