Skip to main content
Filament runs as separate server and control-plane services backed by a datastore, an event bus, and a secret provider. Kubernetes with Helm is the recommended production deployment. Railway is an alternative when Kubernetes worker Jobs are not available.

​
Deployment options

Both options run the server and control plane as separate services and use the same runtime components. The difference is where each run executes.
Run exactly one control-plane replica. Multiple replicas may race while coordinating schedules and runs.

​
Service architecture

Server serves the API and web application. It accepts connection, pipeline, and run requests and persists them through the configured datastore. Control plane manages schedules and coordinates runs. With Kubernetes dispatch it creates worker Jobs and tracks their progress. With inproc dispatch it executes runs inside the control-plane process. The server and control plane coordinate through the datastore and event bus; they do not call each other directly.

​
Where runs execute

The control plane’s DISPATCH_MODE determines where runs execute:
  • kubernetes, the default, creates one worker Job per run. Each run receives its own container and resource configuration. Workers report heartbeats, and the reaper fails runs whose heartbeats become stale.
  • inproc executes runs inside the control-plane process. It requires no worker service, but every run shares the control plane’s CPU and memory. Railway deployments use this mode.
Both modes use the same runner; dispatch changes where it runs, not how a pipeline behaves.

​
Runtime dependencies

A Filament deployment provides three runtime dependencies:
  • Datastore persists run state, checkpoints, connections, pipelines, and schedules. The packaged server and control-plane deployment uses PostgreSQL.
  • Event bus carries run requests and progress events between runtime modules. The packaged deployment uses NATS JetStream.
  • Secret provider stores or resolves connector credentials. The default provider stores encrypted secrets in the configured datastore; AWS Secrets Manager is also supported.
The datastore-backed secret provider requires a base64-encoded AES encryption key. Provider selection and connection settings are covered in Configuration.