> ## Documentation Index > Fetch the complete documentation index at: https://filament.getgalaxy.io/llms.txt > Use this file to discover all available pages before exploring further. # HTTP manifests > Declarative YAML manifests that drive the generic HTTP API source Use an HTTP manifest when an API can be described through endpoints, authentication, pagination, and JSON field mappings. You write YAML instead of a new Go connector. Filament's Notion, Linear, Attio, GitHub, Slack, Resend, Stripe, and PostHog sources all use this path. To try a manifest without recompiling Filament, create an `httpapi` connection and set its required `manifest_path` to your v1 YAML file. Once embedded in the catalog, the same manifest behaves like any other named source. ## Start with the basic structure A manifest has four main parts: connector metadata, user configuration, shared connection behavior, and resources. This shortened GitHub-style example shows how they fit together: ```yaml theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}} version: 1 name: github display_name: GitHub config: token: type: secret required: true help: Personal access token organization: type: string required: true defaults: method: GET query: { per_page: "100" } response: records: $ pagination: { link: next } connection: base_url: https://api.github.com auth: bearer: config.token headers: X-GitHub-Api-Version: "2022-11-28" resources: - name: repositories path: /orgs/{organization}/repos params: { organization: config.organization } primary_key: [id] fields: id: int64 name: string description: string? raw: { path: $, type: json, mode: remainder } capture: { repository: name } - name: issues path: /repos/{organization}/{repository}/issues params: organization: config.organization repository: parent.repository for_each: repositories primary_key: [id] fields: id: int64 title: string updated_at: timestamptz incremental: cursor_field: updated_at start_param: since inject_into: query checkpoint_key: issues_updated_at comparator: time overlap_seconds: 300 ``` ### Config `config` maps field names to `{type, required, default, enum, help, scope}`. Secrets (`type: secret`) render masked and are stored as refs. Config values are referenced elsewhere in the manifest as `config.`. ### Configure requests * `base_url` can reference connector config, as PostHog does for `host`. It is resolved once when the connector is configured, not separately for every request. * `auth` supports no auth, bearer tokens, a custom header, basic auth, and OAuth 2 client credentials. OAuth tokens are cached. Query, HMAC, and chained authentication exist in the Go runtime but cannot currently be declared in YAML. * `rate_limit.requests_per_second` sets a shared limit for the connection. A `dynamic` block can adjust that limit from response headers using a Unix timestamp, seconds from now, or an HTTP date. * `timeout_seconds` sets the request timeout and defaults to 60 seconds. ### Resources Each resource describes one endpoint: * `path` is the URL path. Placeholders such as `{organization}` are filled from `params`, using connection config or values captured from a parent resource. * `primary_key` lists one or more fields that uniquely identify a record. * `fields` maps response values into typed columns. Add `?` to the shorthand type for a nullable field. Use `mode: remainder` to keep unprojected values, or `mode: raw` to keep a complete subtree. * `records` points to the array in the response. Use `cardinality: one` for an endpoint that returns a single object. * `body` describes JSON, form, multipart, raw, or empty request bodies. Linear uses this support for GraphQL requests. Choose the pagination style that matches the API: * `link: ` (RFC 5988 headers, default rel `next`) * `next_url: ` (whole-URL envelopes like Django REST Framework) * `cursor {response, request: query.X|body.X|header.X, more, null_terminates}` * `offset {offset, limit, page_size}` (stops on a short page) * `page {number, size, page_size, total_pages}` ### Read child endpoints Use `for_each: ` when an endpoint must be called once per parent record. The parent's `capture` block names the values available to the child. Child requests default to five concurrent calls, and relationships can nest, such as Resend webhooks → events → attempts. A resource marked `capture_only: true` is walked for its captures and nothing else. It is never emitted, discovered, or selectable, and it only runs when a child that depends on it is selected. Use it when the parent walk a child needs differs from the parent resource users read, for example a full history walk that finds threads with new activity while the readable messages resource stays incremental. A child can gate its fan-out with `parent.since: `. Parents whose captured value is empty are skipped on every run. On an incremental run, parents whose value sorts before the child's effective lower bound under the child's comparator are skipped too, so the child only calls the API for parents that changed since the last run. The child must declare an `incremental` block. ### Incremental An `incremental` block turns a resource into an incremental read: * `cursor_field`: the record field carrying the watermark. It must be projected. * `start_param` + `inject_into: query | body | header`: where the lower bound goes on the request. * `initial`: the first-run lower bound. * `checkpoint_key`: the durable key (defaults to `cursor_field`). * `comparator: lex (default) | numeric | time`. * `overlap_seconds` rewinds the lower bound for a lookback window and is valid for `time` **or** `numeric` comparators. The lower bound remains fixed during one extraction. Filament saves the largest cursor value it sees. A missing or invalid cursor fails the run because skipping that value could cause later runs to miss records. ### Discovery and streaming `discovery` supports `mode: static | dynamic`, though every shipped manifest is static. A `stream` mode (`ndjson | sse | chunked_array`) exists per resource as an alternative to pagination. No shipped manifest uses it. ## Runtime behavior **Available modes.** Every manifest can run a full read. Incremental reads are available only when at least one resource has an `incremental` block, which is why Attio and Resend are full-only. The manifest chooses the cursor field; a pipeline can change only its lookback window. **Resuming full reads.** Top-level resources checkpoint their pagination cursor per page, so an interrupted full run resumes mid-listing. Child (fan-out) resources always restart from the beginning. A resumed cursor the API no longer honors falls back to a full read with a warning rather than failing. **Incremental child resources.** Children share one watermark per resource across all parents. There is no per-parent checkpoint. If parents progress at different speeds (say, per-repository issues), a run can advance the watermark past rows in slower parents. Set `overlap_seconds` generously to re-read the window and let an upsert write mode absorb the duplicates. **Retries and limits.** Responses are handled by status: * HTTP 429 is retried up to 10 attempts, honoring `Retry-After` capped at 60 seconds and reported as `pressure.rate_limited` * 5xx responses are retried 3 times * other 4xx responses are fatal Responses are capped at 100 MB. Top-level resources extract concurrently, children after their parents. **Connection testing.** When a caller explicitly invokes the connector's `TestConnection` capability, it sends one request to the first top-level, non-streaming resource. Saving or validating a connection through the service API does not invoke this live test. **Manifest validation.** Loading a manifest reports all detectable problems together. It checks: * required objects, field types, and allowed enum values * strict YAML decoding that rejects unknown fields * defaults and shorthand expansion * semantic checks for cross-field rules such as parent cycles and unprojected cursor fields ## Maturity Manifest-driven integrations start at `alpha`. Their behavior is derived from API documentation, and schema validation plus mocked HTTP tests prove internal consistency, not fidelity to the live API. They graduate to `beta` once run end-to-end against the real service.