Read Connector

Create New Read Connector

After you have an Ascend Read Connection set up, then you continue to set up the Read Connector in your dataflow.

Figure 1Figure 1

Figure 1

In Figure 1 above:

CONNECTOR INFO

  • Name (required): The name to identify this connector with.
  • Description (optional): Description of what data this connector will read.

Connector Configuration

Figure 2Figure 2

Figure 2

You can either manually provide Object Pattern(required), which is the directory of the files you are looking for or click on Browse and Select Data: this button allows to explore resource and locate assets to ingest. This will give you access to the SFTP Server database in a modal dialog (Figure 3 and 4 below), where you can navigate to the desired table to be imported. Select the file or files you want to ingest and press Select.

Select the Parser(required) that match the data. If needed, some other advanced options are shown such as Field Delimiter and Date Format for CSV files, as shown in Figure 2

Figure 3Figure 3

Figure 3

Figure 4Figure 4

Figure 4

Generate Schema

Once you click on the GENERATE SCHEMA button, the parser will create a schema and a data preview will be populated as in the Figure 5 below.

  • Add schema column: Add a custom column to the generated schema
Figure 5Figure 5

Figure 5

Component Pausing

Update the status of the read connector by marking it either Running to mark it active or Paused to pause the connector from running.

Figure 6Figure 6

Figure 6

Refresh Schedule

The refresh schedule specifies how often Ascend checks the data location to see if there's new data. Ascend will automatically kick off the corresponding big data jobs once new or updated data is discovered.

Figure 7Figure 7

Figure 7

Processing Priority (optional)

When resources are constrained, Processing Priority will be used to determine which components to schedule first.

Figure 8Figure 8

Figure 8

Higher priority numbers are scheduled before lower ones. Increasing the priority on a component also causes all its upstream components to be prioritized higher. Negative priorities can be used to postpone work until excess capacity becomes available.


Did this page help you?