Write Connector

Create New Write Connector

In Figure 1 above:

CONNECTOR INFO

• Name (required): The name to identify this connector with.
• Description (optional): Description of what data this connector will write.

Connector Configuration

You can either manually provide Container Name, which is the container are writing or click on Browse and Select Data: this button allows to explore resource and locate the destination. Select the destination you want to write and press Select.

• OUTPUT DIRECTORY(required): The output prefix to write files to, e.g. 'data/gold/aggregated_cab_rides'. Note: This prefix must be unique across all Write Connectors and Ascend will delete any existing objects in the directory.
• PARTITION INTERPOLATION TEMPLATE(optional): Include a value from the partition profile as part of the output directory naming. For example, to create Hive style partitioning on dataset daily partitioned on timestamp 'event_ts', specify the pattern as 'dt={{event_ts(yyyy-MM-dd)}}/'.
• OUTPUT FILE SUFFIX(optional): A suffix to attach to each file name. By default, Ascend will include the extension of the file format, but you may optionally choose a different suffix.
• FORMAT(required): list of all formats of the destination files: Avro, CSV, JSON, ORC, Parquet, Text.
• PATH DELIMITER(optional): The delimiter to use in joining path segments, defaults to '/'.
• MANIFEST FILE(optional): Specify a manifest file which will be updated with the list of files every time they are updated.
• WRITE STRATEGY(optional): Pick the strategy for writing files in the storage:
  • Default (Mirror to Blob Store): this strategy allows to keep the storage aligned with ascend. allows inserting, updating and deleting partitions on the blob store.
  • Ascend Upsert Partitions: This strategy allows for appending new partitions in Ascend and updating existing partitions, without deleting partitions from blob store that are no longer in Ascend.