Runs

A run is one execution of one source-to-sink route from an immutable pipeline version. Its messages are defined in protos/ingestion/v1/runs.proto.

​

RPCs

​

RunPipeline

One call may produce several runs. A graph that fans one source out to two sinks has two routes, so the response contains two edge_runs, each keyed by its pipeline_edge_key. client_token distinguishes retry from intent. Reuse the token when retrying the same user action; choose a new token for a deliberate rerun. The token contributes to stable run IDs, preventing a transport retry from dispatching duplicates. worker_configuration overrides pipeline worker settings field by field. The resolved configuration is copied into each run so later pipeline edits cannot change work already requested. A zero value asks the engine or sink to choose its normal default:

​

Status and recovery

PARTIAL does not mean that any checkpoint happened to exist. Every resource in the route must meet the safe-resume rules. See Runs and recovery for those boundaries.

​

ListRuns

Filter by pipeline_id, optional pipeline_version_id, repeated status, and a since_ms/until_ms window over started_at (inclusive/exclusive epoch milliseconds). Windowed queries omit runs that have not started. Search and sorting fields follow the common API conventions, and list pagination is described in Service overview. Scheduled pipelines normally show future SCHEDULED rows. The scheduler creates the next run in advance and promotes it to requested at fire time.

​

GetRun

Returns RunSnapshot with run-level status, record and byte totals, errors, lifecycle timestamps, connection IDs, worker usage, and per-resource progress. The protobuf also contains at_seq, but the current GetRun implementation does not populate it, so clients should treat it as zero/reserved rather than as a snapshot consistency token.

​

TailRun

TailRun is the server-streaming RPC. Each RunEvent includes its wire event type, resource, monotonic sequence, and relevant measurements. With should_replay, the server first synthesizes events from stored resource progress and marks them is_replay: true, then forwards live facts. This helps a reconnecting client render current progress before new events arrive. The stream closes after a terminal or paused/canceled event. An event bus is required; without one the RPC returns failed_precondition.

​

SignalRun

Send PAUSE, RESUME, or CANCEL with a run_id. An optional tenant_id is checked against the run when present. The allowed transition depends on current state:

• requested runs can be paused or canceled without worker coordination;
• running pause/cancel requests wait for the worker to acknowledge its safe commit or abort boundary;
• paused and partial runs can be resumed to requested or canceled; and
• repeated signals that already match the intended state are idempotent where safe, while invalid transitions return failed_precondition.

Running signals require both an event bus and datastore support for atomic run transitions. A request may also end with deadline_exceeded or canceled while waiting for worker acknowledgement.