Skip to main content
Filament’s performance work lives in the dedicated Galaxy benchmarks repository. The repository contains the harness, datasets, system adapters, methodology, and raw results. Keeping that work separate from Filament’s implementation makes a benchmark easier to reproduce and ties each result to the exact environment and configuration that produced it.
The benchmark program is in progress. Current results describe specific full-load cohorts, not a general performance guarantee for every connector, replication mode, dataset, or deployment.

​
What is measured

The current data-movement benchmark measures complete dataset loads. It uses TPC-H or NYC Taxi data and supports these routes:
The harness compares Filament with other replication tools and, where applicable, native database dump-and-load utilities. The authoritative list of systems and supported routes lives in the benchmark repository so additions do not require a Filament docs release. This benchmark does not currently measure incremental catch-up, steady-state CDC throughput, or recovery after failure.

​
How a comparison is run

A group of comparable results uses the same prepared source data. Each repetition receives a fresh destination, and remote SQL endpoints can be restarted and allowed to settle before timing begins. The harness rotates tool order across repetitions so one tool does not always run first or last. Setup work such as provisioning, dataset generation, source loading, image pulls, and destination validation is outside the measured transfer window. Process initialization triggered by the transfer remains inside it. A result is recorded only when every destination table matches the source row count. Failed or incomplete transfers do not become benchmark results.

​
Evidence recorded with each result

Raw result files include more than elapsed time:
  • Dataset, route, topology, and repetition count
  • Tool configuration and container image identity
  • Benchmark host and source revision
  • CPU, peak memory, and network use for measured containers
  • Destination row-count validation
  • Database and CloudWatch measurements when the remote environment provides them
This context matters because results from different machines, endpoint sizes, topologies, configurations, or benchmark commits are not directly comparable.

​
Limits of the results

For example, the harness measures Debezium’s initial snapshot in this scenario, not the steady-state CDC workload for which Debezium is primarily designed.

​
Results and reproduction

The data-movement benchmark documents supported tools, routes, datasets, prerequisites, and commands. Its methodology defines the timed window, correctness checks, reference environment, and result format. Dated result directories contain the raw JSON for each completed cohort.