Blob Storage Write Connector Formats
Of the available parsers for some Write Connectors, the following have additional required or optional properties.
CSV Format
The CSV format provides a range of customizable parameters. Each parser for a Write Connector has specific parameters.
All fields available for the CSV format are optional. We'll use the following text example to discuss each field:
@Album released in 1967
artist,album,track,verse
The Beatles, Magical Mystery Tour, "Hello, Goodbye", "You say, /"Yes/", I say, /"No/" !/I say, /"Yes/",
but I may mean, /"No/"!/"
| Field | Description |
|---|---|
| A SINGLE CHARACTER AS A SEPARATOR FOR EACH FIELD AND VALUE | Field delimiter to be used. Ex. Delimiter = , |
| QUOTE CHARACTER | A single character used for escaping quoted values where the separator can be part of the value. If you would like to turn off quotations, you need to set the value to an empty string. Ex. track = "Hello, Goodbye" (In this example, the quote character is ".) |
| ESCAPE CHARACTER | A single character used for escaping quotes inside an already quoted value. Ex. verse1= "You say, /"Yes/", I say, /"No/" !/I say, /"Yes/", but I may mean, /"No/"!/" (In this example, the escape character is /.) |
| CHAR TO ESCAPE QUOTE ESCAPING | Sets a single character used for escaping the escape for the quote character. Ex. verse1= "You say, /"Yes/", I say, /"No/" !/I say, /"Yes/", but I may mean, /"No/"!/"* (In this example, the lyrics utilize a forward slash to denote the addition of background lyrics. Adding an exclamation mark (!) before the forward slash will preserve the forward slash in the string. The character to escape quote escaping is !.) |
| Escape Quotes | Indicates whether values containing quotes should always be enclosed in quotes. Leaving deselected results in writing with the default behavior of escaping all values containing a quote character. |
| Quote All | Indicates whether all values should always be enclosed in quotes. Leaving deselected results in writing with the default behavior of escaping values containing a quote character. |
| Files Have a Header Row | Writes the names of columns as the first line. Leave this unchecked if the first row does not contain column names. Ex. Header Row = artist,album,song,verse |
| Null Value | Sets the string representation of a null value. |
| Empty Value | Sets the string representation for empty values. Defaults to "". Select Custom to set a specific representation. |
| Encoding | Specify CSV files by the given encoding type. Defaults to UTF-8 (this is the most common encoding type). One known exception is files created on Windows, which requires UTF-16, instead. |
| Compression | Compression codec to use when saving to file. Options are bzip, deflate, gzip, Iz4, and snappy. |
| Date Format | Format of the date type field in the CSV file. See Datetime Patterns for Formatting and Parsing. |
| Timestamp Format | Format of the timestamp type field in the CSV file. Specify the timestamp format if known. See Datetime Patterns for Formatting and Parsing. |
| Ignore Leading White Space | Indicates whether or not leading whitespaces from values being written should be skipped. (Default behavior will include leading white spaces.) |
| Ignore Trailing White Space | Indicates whether or not trailing whitespaces from values being written should be skipped. (Default behavior will incldue leading white spaces.) |
| Line Separator | Defines the line separator that should be used for parsing. Defaults to \n (the newline character). |
JSON Format
All fields for the JSON format are optional.
| Field | Description |
|---|---|
| Compression | |
| Date Format | Format of the date type field in the CSV file. See Datetime Patterns for Formatting and Parsing. |
| Timestamp Format | Format of the timestamp type field in the CSV file. Specify the timestamp format if known. See Datetime Patterns for Formatting and Parsing. |
| Encoding | Specify JSON files by the given encoding type. Defaults to UTF-8 (this is the most common encoding type). One known exception is files created on Windows, which requires UTF-16, instead. |
| Ignore Null Fields | Indicates whether or not to ignore null fields when generating JSON objects. (Default behavior will include null fields.) |
Updated about 2 years ago