Google Cloud Storage
Creating a Google Cloud Storage (GCS) Connector
- Access credentials
- Data location on Google Cloud Storage
- Data Schema (column names and column type)
Specify the Google Cloud Storage (GCS) path
GCS Read Connectors have location settings comprised of:
- Bucket: The bucket name, such as
- Pattern: The pattern used to identify eligible files:
Google Cloud Storage Credentials
Enter the JSON key for the service account which has write access to everything matching the GCS path. Refer to Google documentation for more details on GCS authentication.
Use Test Connection to check whether all GCS permissions are correctly configured.
Parsers & Schema
Data formats currently available are: Avro, Grok, JSON, Parquet and XSV. However, you can create your own parser functions or define a UDP (User Defined Parser) to process a file format.
Schema information will automatically be fetched for JSON, Parquet and XSV files with a header row.
Updated almost 4 years ago