> ## Documentation Index
> Fetch the complete documentation index at: https://filament.getgalaxy.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Pipeline Throughput

> BenchmarkNullSinkPipeline and its parallel variant.

Defined in `pipeline/pipeline_bench_test.go`.

## `BenchmarkNullSinkPipeline`

The top rung of the [subtraction ladder](/pages/benchmarks/methodology/introduction):
20,000 \~80-byte JSON records, built once during setup so fixture cost isn't
measured, pushed through a pipeline configured with `BatchMaxRows: 1000`
into a `nullSink`. `FlushInterval` is set to an hour so row-count and
`Close` drive batching rather than the timer — the benchmark measures
throughput, not flush timing.

```sh theme={"theme":"vitesse-black"}
go test -bench=BenchmarkNullSinkPipeline -benchmem ./pipeline/...
```

Reports `records/sec` (computed from `b.Elapsed()`) alongside standard
allocs/op.

## `BenchmarkNullSinkPipelineParallel`

The same idea with concurrency: 40,000 records spread across 8 resources
(so the batcher's sharding engages), pushed from a pool of goroutines
sized to match, at parallelism levels 1, 4, and 8 (`b.Run` sub-benchmarks).
This isolates how the pipeline itself scales — independent of the
shared-database contention a real end-to-end benchmark against an actual
sink would also be measuring.

```sh theme={"theme":"vitesse-black"}
go test -bench=BenchmarkNullSinkPipelineParallel -benchmem ./pipeline/...
```

## Why a null sink

`nullSink` still does real integrity work — it recomputes the write-side
CRC exactly as a production sink would via `filament.CRC32C` — it just
discards the batch afterward instead of writing it anywhere, so memory
stays flat across `b.N` iterations. That keeps the benchmark honest about
the pipeline's own cost (batching, the read-side CRC, integrity
verification) without a real destination's I/O in the number.