Copy/Paste Components

The ability to copy components within a Dataflow and paste them into other Dataflows greatly speeds up development since you're no longer starting from scratch each time. The copied elements are represented as JSON files, making them easy to store in repositories such as GitHub for efficient Dataflow versioning and tracking.

Selecting Components to Copy

You can select components to copy via the User Interface in two ways - through the Browse menu or the Build process. Though the copy/paste process is the same, the selection method differs slightly. Here, we'll focus on using the Browse menu.

When the Browse menu is open, your screen should resemble the following:

To choose individual components, shift + select the components you want to copy. A check-box will appear on the selected component, and it will also be highlighted in the list within the tool pane.

🚧

Please be aware that Subscribed Data Feeds and Groups in the Dataflow are not copied and will need to be recreated post-paste as necessary.

Once you've selected the necessary components, a series of option boxes will appear above the component list. Click the "Copy" box to start the copying process.

Copying Components

The Copy Components pane provides a JSON representation of all the components in the Dataflow or those you've selected individually. Scroll to the bottom of the JSON text to find the "Copy to Clipboard" and "Download" buttons. In our example, we'll use the "Copy to Clipboard" function.

Create a new Dataflow and select Build from the left-hand menu bar. Three options will be presented at the top of the left-hand pane: Create, Paste, and Copy. Choose Paste.

Position your cursor in the box next to the "1" and paste the copied JSON. Then, hit the "Paste" button at the bottom of the modal beneath your JSON.

Follow the sequence of prompts at the bottom of the left-hand pane to confirm the components you are pasting, confirm the necessary credentials, and initiate the pasting process by clicking the "Start Pasting" button at the top of the right-hand modal. Upon completion, click the "Finish" button at the bottom of the right-hand modal.

💡

When pasting to a different Data Service, ensure the necessary connections/credentials are shared to facilitate a smooth transition.

Your newly pasted Dataflow should now be displayed in the right-hand modal, with all components instantly updated. Ascend recognises the duplicate data from the original Dataflow and ensures it's up-to-date immediately. You can then proceed with further development without any need for data processing to back-fill your new Dataflow.

🚧

Caution

Exercise caution when pasting a Write Connector to avoid:

1. Creating a duplicate write connector, both pointing to the same warehouse or blob store directory.
2. Ensure the Write Connector target doesn't contain any pre-existing data, as Ascend will delete anything at the target that doesn't belong to the Write Connector.