Blob Storage Write Connector Formats

Of the available parsers for some Write Connectors, the following have additional required or optional properties.

CSV Format

The CSV format provides a range of customizable parameters. Each parser for a Write Connector has specific parameters.

All fields available for the CSV format are optional. We'll use the following text example to discuss each field:

@Album released in 1967 
artist,album,track,verse
The Beatles, Magical Mystery Tour, "Hello, Goodbye", "You say, /"Yes/", I say, /"No/" !/I say, /"Yes/", 
but I may mean, /"No/"!/"
FieldDescription
A SINGLE CHARACTER AS A SEPARATOR FOR EACH FIELD AND VALUEField delimiter to be used.

Ex. Delimiter = ,
QUOTE CHARACTERA single character used for escaping quoted values where the separator can be part of the value. If you would like to turn off quotations, you need to set the value to an empty string.

Ex. track = "Hello, Goodbye"
(In this example, the quote character is ".)
ESCAPE CHARACTERA single character used for escaping quotes inside an already quoted value.

Ex. verse1= "You say, /"Yes/", I say, /"No/" !/I say, /"Yes/",
but I may mean, /"No/"!/"

(In this example, the escape character is /.)
CHAR TO ESCAPE QUOTE ESCAPINGSets a single character used for escaping the escape for the quote character.

Ex. verse1= "You say, /"Yes/", I say, /"No/" !/I say, /"Yes/",
but I may mean, /"No/"!/"*
(In this example, the lyrics utilize a forward slash to denote the addition of background lyrics. Adding an exclamation mark (!) before the forward slash will preserve the forward slash in the string. The character to escape quote escaping is !.)
Escape QuotesIndicates whether values containing quotes should always be enclosed in quotes. Leaving deselected results in writing with the default behavior of escaping all values containing a quote character.
Quote AllIndicates whether all values should always be enclosed in quotes. Leaving deselected results in writing with the default behavior of escaping values containing a quote character.
Files Have a Header RowWrites the names of columns as the first line. Leave this unchecked if the first row does not contain column names.

Ex. Header Row = artist,album,song,verse
Null ValueSets the string representation of a null value.
Empty ValueSets the string representation for empty values. Defaults to "". Select Custom to set a specific representation.
EncodingSpecify CSV files by the given encoding type. Defaults to UTF-8 (this is the most common encoding type). One known exception is files created on Windows, which requires UTF-16, instead.
CompressionCompression codec to use when saving to file. Options are bzip, deflate, gzip, Iz4, and snappy.
Date FormatFormat of the date type field in the CSV file. See Datetime Patterns for Formatting and Parsing.
Timestamp FormatFormat of the timestamp type field in the CSV file. Specify the timestamp format if known. See Datetime Patterns for Formatting and Parsing.
Ignore Leading White SpaceIndicates whether or not leading whitespaces from values being written should be skipped. (Default behavior will include leading white spaces.)
Ignore Trailing White SpaceIndicates whether or not trailing whitespaces from values being written should be skipped. (Default behavior will incldue leading white spaces.)
Line SeparatorDefines the line separator that should be used for parsing. Defaults to \n (the newline character).

JSON Format

All fields for the JSON format are optional.

FieldDescription
Compression
Date FormatFormat of the date type field in the CSV file. See Datetime Patterns for Formatting and Parsing.
Timestamp FormatFormat of the timestamp type field in the CSV file. Specify the timestamp format if known. See Datetime Patterns for Formatting and Parsing.
EncodingSpecify JSON files by the given encoding type. Defaults to UTF-8 (this is the most common encoding type). One known exception is files created on Windows, which requires UTF-16, instead.
Ignore Null FieldsIndicates whether or not to ignore null fields when generating JSON objects. (Default behavior will include null fields.)