Exporting component(s) within a Dataflow and importing them into other Dataflows allows for quick development as you don't have to build them from scratch each time. Furthermore, exports are simple JSON files that can be pushed into repositories such as GitHub for Dataflow versioning and tracking.
There are 2 primary ways to export components via the UI. One is via the Browse menu and other thru build. They use the same exact export/import process they just vary on how components are selected for export. Here we will focus on using the Browse menu.
With the Browse menu open, your screen should look like this:
To select individual components hold the SHIFT KEY and click on the components that you want to exported. You will see a check-box on the component as well as the component selected in the list on the left-side modal.
Subscribed Data Feeds and Groups in the Dataflow will not be exported and will need to be recreated as needed post import.
From here, you will see a number of options represented by boxes above the component list. One will be "Export". Click on that box to begin the Export process.
In the Export Components pane, you will find the JSON representation of all the components in the Dataflow or the ones you selected individually. Scroll to the bottom of the JSON to find the "Copy to Clipboard" and "Download" buttons. In our example we are going to copy to clipboard.
Now the fun starts! Create your new Dataflow, and from the left hand menu bar select Build. From there your left hand modal will have 3 options at the top; Create, Import, and Export. Select Import.
Place your cursor in the box to the right of the "1", and paste your copied JSON. Now select the "Import" option at the bottom of the modal underneath your JSON. At this point you should see a screen like this:
From this point follow the series of prompts at the bottom of the left-hand modal, to confirm the components you are important, confirm credentials, and finally start the import process by selecting the "Start Import" button at the top of the right-hand modal. When your Import is complete select the "Finish" button at the bottom of the right-hand modal!
At this point your new Dataflow should be showing on the right-hand modal, with all components immediately up-to-date! That's because Ascend knows to de-duplicate the data underneath since the code is exactly the same as the Exported Dataflow. You can immediately start building/iterating without having to process any data to back-fill your new Dataflow!
Use caution when importing a Write Connector to avoid:
1.) Creating a duplicate write connector, both pointing to the same warehouse, blob store directory, etc.
2.) Settings for the Write Connector target does not contain any pre-existing data, as Ascend will delete anything at the target that doesn't belong to the Write Connector.
Updated 2 months ago