Skip to main content
Start with the change you need to observe, then choose how the destination should represent it. Filament validates that combination before data moves.

​
Choose a read mode

An incremental users route might use updated_at as its cursor. That is lighter than rereading the table, but it cannot see a hard delete. Use CDC when deletes or source transaction order must propagate. A source connection is configured as either standard or CDC. Standard routes choose full or incremental per resource. CDC applies to the whole route and cannot be mixed with standard resources in the same run.

​
Choose destination behavior

  • append keeps each emitted row version.
  • replace produces a new snapshot of the resource.
  • upsert keeps the current row for each primary key.
  • merge applies ordered CDC inserts, updates, and deletes.
Full reads default to replace. Incremental reads default to upsert. CDC routes default to append but may be configured to merge; the choice is route-wide. Upsert and merge need primary-key information, and Filament rejects the plan when the source cannot supply it. Common combinations look like this:

​
Reference: resolved modes

You can skip this section unless you use the API or build a connector. Internally, each read/write pair becomes an ingestion type such as full_replace, incremental_upsert, cdc_append, or cdc_merge. That plan sets the sink mode, key requirement, and checkpoint timing. You usually need the names only when using the API or building a connector. Supported combinations are: The engine checks the source policy, sink capabilities, primary keys, and cursor configuration while resolving the plan. A connector advertising a mode does not bypass those runtime contracts.

​
How incremental upserts choose the latest row

For an incremental upsert, the cursor tells the source where the next read begins. The sink also uses that field to decide which version of a row is newer. If you do not choose a cursor, Filament can use the connector’s recommendation. Full upserts use insert order instead of a cursor. A checkpoint controls where reading resumes; the version rule controls which row wins at the destination.

​
Columns Filament adds

Filament reserves the _filament_ prefix and adds the logical run ID and start time to replicated rows. CDC also adds the operation, source position, and sequence needed to interpret the change stream. These fields are nullable so an existing destination can evolve without rewriting older rows. A source column using a reserved name is rejected rather than overwritten.