- A connection says how to reach a source or sink. Credentials and connector settings live here so they can be reused.
- A pipeline connects sources to sinks. Each edge is a route with its own replication choices. Saving an edit creates a new immutable version.
- A resource is one table, stream, or API collection selected from a source.
- A run is one execution of one route.
- A checkpoint records durable progress for a resource that can resume.
users table moving from PostgreSQL to a warehouse. The connection
answers “which PostgreSQL?” and “which warehouse?” The route answers “copy all
users or only changes?” A run is the attempt that performs that work. If the
chosen mode supports checkpoints, a retry can continue from confirmed progress.
The data journey
During a run, Filament:- Resolves the two connectors and the selected resources;
- Checks that the requested read and write behavior is supported;
- Asks the source to write typed rows into Arrow builders;
- Sends bounded Arrow batches to the sink;
- Commits or aborts according to the sink contract; and
- Records facts, totals, integrity evidence, and durable checkpoints.
Choose what to learn next
- Replication modes helps you choose between full, incremental, and CDC reads and their write behavior.
- Architecture follows a batch through the engine and explains backpressure and parallelism.
- Integrity and checkpoints separates data verification from recovery progress.
- Runs and recovery explains states, retries, pause, resume, and cancellation.