go test -bench), living
alongside the code they measure — for example
pipeline/pipeline_bench_test.go and eventbus/inproc/inproc_bench_test.go.
The subtraction ladder
Rather than one end-to-end number, the pipeline benchmarks build a ladder: measure a component in isolation, then measure it with one more real piece added, so each rung’s cost is attributable. The top rung,BenchmarkNullSinkPipeline, pushes records through the full
batcher → writer → integrity loop into a zero-I/O nullSink — no source
read, no event bus, no real storage. It still recomputes the write-side CRC
exactly as a real sink would, so the integrity work stays in the
measurement; it just discards the batch instead of persisting it. That makes
it the pipeline’s own cost floor: whatever a real end-to-end (engine)
benchmark costs above this rung is source read, event bus, and storage —
not the pipeline itself.
BenchmarkNullSinkPipelineParallel extends this by sharding records across
multiple resources and pushing from a pool of goroutines, sweeping
parallelism (1, 4, 8) to see how the pipeline scales once batching and the
read-CRC step are no longer single-goroutine — isolated from the
shared-database contention a real macro/engine benchmark would also be
measuring.
Both report a custom records/sec metric alongside Go’s standard
allocs/op, so throughput and allocation cost are visible side by side.
Running the benchmarks
What’s covered so far
Pipeline throughput
The subtraction-ladder benchmarks in detail.
Event bus overhead
Publish cost with no subscribers, fan-out, and contention.
This section documents what’s measured and how to reproduce it, not
published numbers — results depend on hardware and haven’t been captured
here yet. Run the commands above to get numbers for your own machine.