- Access credentials
- Data location on Google Cloud Storage
- Data Schema (column names and column type)
GCS Read Connectors have location settings comprised of:
- Bucket: The bucket name, such as
- Pattern: The pattern used to identify eligible files:
Enter the JSON key for the service account which has write access to everything matching the GCS path. Refer to Google documentation for more details on GCS authentication.
Use Test Connection to check whether all GCS permissions are correctly configured.
Data formats currently available are: Avro, Grok, JSON, Parquet and XSV. However, you can create your own parser functions or define a UDP (User Defined Parser) to process a file format.
Schema information will automatically be fetched for JSON, Parquet and XSV files with a header row.
Updated about 3 years ago