> ## Documentation Index
> Fetch the complete documentation index at: https://filament.getgalaxy.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Google Cloud Storage

> Write NDJSON, JSON, or Parquet objects to Google Cloud Storage

The Google Cloud Storage sink (**alpha**) writes each resource as one NDJSON,
JSON, or Parquet object per run. It uses the same run-versioned layout and
success-manifest contract as the S3 sink.

## Configuration

| Field                | Scope      | Default                           | Description                                                            |
| -------------------- | ---------- | --------------------------------- | ---------------------------------------------------------------------- |
| `bucket`             | Connection | —                                 | Required. Destination GCS bucket.                                      |
| `auth_method`        | Connection | `application_default_credentials` | Uses the attached service account, GKE Workload Identity, or local ADC |
| `prefix`             | Pipeline   | —                                 | Root folder for the pipeline, with surrounding `/` trimmed             |
| `partition`          | Pipeline   | `dt={{.Date}}`                    | Folders between each resource and its files                            |
| `file_format`        | Pipeline   | `ndjson`                          | `ndjson`, `jsonl`, `json`, or `parquet`                                |
| `compression`        | Pipeline   | per format                        | `none` or `gzip` for JSON formats; `snappy` for Parquet                |
| `chunk_size_mib`     | Pipeline   | `16`                              | Resumable-upload chunk size in MiB, from 1 through 1024                |
| `upload_concurrency` | Pipeline   | `4`                               | Concurrent GCS operations across resources, from 1 through 32          |

Authentication uses Google Application Default Credentials. On GKE, use
Workload Identity; for local development, run
`gcloud auth application-default login` before starting Filament. The sink
does not require a project or region setting because a GCS bucket name is
globally unique and the bucket already owns its location configuration.

`TestConnection` reads bucket metadata. Runs require permission to create
objects in the configured bucket.

## Object layout

With prefix `local_pg`, resource `accounts`, and the default partition, a run
writes:

```text theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
local_pg/accounts/dt=2026-09-10/<run>.parquet
local_pg/_runs/<run>/_SUCCESS.json
```

The partition template supports `{{.Date}}`, `{{.StartedAt}}`,
`{{.Resource}}`, and `{{.Run}}`. Values use the run's UTC start time. Leave the
partition empty to write objects directly under the resource directory.

The success manifest lists every resource's key, `gs://` URI, row count, byte
count, and CRC32C. Consumers should treat `_SUCCESS.json` as the visibility
boundary and ignore runs without it.

## Write mechanics and failure behavior

Each resource is streamed through a GCS resumable writer. The client buffers
one configured chunk per active resource and retries transient chunk failures.
Objects are finalized during `Commit`; the success manifest is uploaded only
after all resource writers close successfully.

`Abort` cancels unfinished resumable writers. If `Commit` fails after some
resource objects have finalized, those objects can remain without a success
manifest, matching the S3 sink's resource-level atomicity.