Skip to content
Connect

Google Cloud Storage connection strings

Use the gs:// scheme for Google Cloud Storage (GCS). Connect can rely on Application Default Credentials (ADC), service account JSON files, or credentials stored in aliases.

Standard buckets

Section titled “Standard buckets”
  • gs://project-exports — bucket root using ADC (gcloud login, workload identity, etc.).
  • gs://project-exports/daily/ — object prefix inside the bucket.

Service account keys

Section titled “Service account keys”
  • gs:///var/secrets/gcs-writer.json@project-exports/daily/ — absolute path to a JSON key file (three slashes because the path begins with /).
  • gs://keys/gcs-reader.json@project-exports/reports/ — key file located relative to the current working directory.

Inline credential payloads

Section titled “Inline credential payloads”
  • JSON content directly in the URI:
    gs://user:{"type":"service_account","project_id":"proj",...}@bucket/path (URL-encode special characters).
  • Base64 encoded JSON (ideal for CI secrets):
    gs://user:eyJ0eXBlIjoi...@bucket/path.

Signed URLs and impersonation

Section titled “Signed URLs and impersonation”
  • gs://impersonate:target-service-account@project-exports/delta/ — use workload identity federation or impersonation (combine with ADC credentials that have roles/iam.serviceAccountTokenCreator).

Absolute vs relative prefixes

Section titled “Absolute vs relative prefixes”
  • gs://project-exports//tmp/uploads/ — preserves the leading slash in the object key (/tmp/uploads/...).
  • gs://project-exports/uploads/ — stores objects under uploads/ in the bucket.

Putting it together

Section titled “Putting it together”
Terminal window
$ connect move --parallel 3 \
    /data/outgoing/*.csv \
    gs:///etc/connect/keys/gcs-writer.json@project-exports/incoming/

The command authenticates with the JSON key at /etc/connect/keys/gcs-writer.json and moves CSV files into the incoming/ prefix.

More usage patterns:

  • gs://project-exports/raw/*.parquet — rely on gcloud auth application-default login credentials for discovery.
  • gs://user:${GCS_CREDS_B64}@archive-bucket/regex:^2024-.*\\.gz$ — read base64-encoded credentials from the GCS_CREDS_B64 environment variable.
  • gs://user:/opt/keys/source.json@bucket-a/file.txt to gs://user:/opt/keys/dest.json@bucket-b/file.txt — use different service accounts per bucket in a single transfer.