Read Connector

After creating an IBM DB2 on AS400 connection, set up the Read Connector in your dataflow.

Create a new Read Connector

Step 1: Find your newly created connection and select USE.

12171217

Figure 1.

Step 2: Browse and select the data to ingest. If you know the path of the data you want to ingest, select "Skip and configure manually."

13621362

Figure 2.

Step 3: Provide the required Connector information.

13481348

Figure 1.

FieldRequiredDescrtiption
NameRequiredProvide a name for your connector. We recommend using lowercase with underscores in place of spaces.
DescriptionOptionalDescribes the connector. We recommend providing a description if you are ingesting information from the same source twice, for different reasons.
Table NameRequiredName of the table being ingested. This can either be supplied manually or selected in Step 2.
Schema NameOptionalIf using a specific schema configuration, provide a name.

Schema

Once you click on the GENERATE SCHEMA button, the parser will create a schema and a data preview will be populated as in the Figure 5 below.

  • Add schema column: Add a custom column to the generated schema
935935

Figure 5

Refresh Schedule

The refresh schedule specifies how often Ascend checks the data location to see if there's new data. Ascend will automatically kick off the corresponding big data jobs once new or updated data is discovered.

Component Pausing

Update the status of the read connector by marking it either Running to mark it active or Paused to pause the connector from running.

Advanced Settings

Processing Priority: When resources are constrained, will be used to determine which components to schedule first.

Custom Spark Params: You can determine the resources such as driver size, number of executors, and the executor size

Higher priority numbers are scheduled before lower ones. Increasing the priority on a component also causes all its upstream components to be prioritized higher. Negative priorities can be used to postpone work until excess capacity becomes available.