A Data Feed communicates live data across different Dataflows in Ascend, even if they are in different Data Services. One Dataflow "publishes" a Data Feed that other Dataflows subscribe to. These two ends are linked together live, such that if any new data is processed, it will be automatically reflected to all subscribers -- and the components linked downstream will start processing as well.
You can create new Data Feeds (publish) or connect to existing Data Feeds (subscribe).
The permissions of a Data Feed determine which Data Services and teams / individuals are allowed to subscribe to this Data Feed. In this way, Data Feeds allow for sharing of data with strict governance and access controls managed by Ascend.
There are two permission options:
- Allow all Teams in all Data Services to access this Data Feed: This option allows every user / team in the system to see and subscribe to the Data Feed.
- Only selected teams in selected Data Services can access this Data feed: This options allows for fine-grain customization of access to the Data Feed.
- Click Share in the left sidebar menu
- Click + NEW DATA FEED in the top right corner of the section.
- Provide a name for the Data Feed. As a best practice, this name should clearly describe the nature of the data. Optionally provide a description.
- Select the upstream transform for the Data Feed. From the drop-down you'll see a list of all the transforms in your Dataflow to choose from. Alternatively, you can right-click the Transform you want to create a Data Feed Connector from, and select Create new data feed.
- Select the appropriate Permissions for this Data Feed.
- Click Create to finish creating the Data Feed.
Once a Data Feed is created, users with the required permission are able to subscribe to it Connecting to an existing Data Feed Connector.
If one or more Data Feed Connectors are subscribing to a Data Feed, you won't be able to delete the Data Feed until those Data Feed Connectors are all deleted first.
- Navigate to the BUILD panel and click SHARE.
- This view displays a catalog of Data Feeds. You can search by name and use Filter By to help narrow your search.
- Select the Data Feed you would like to subscribe to, view the preview, and click SUBSCRIBE.
- The Dataflow adds the subscription and this Component is now available for further transformation.
A Data Feed Connector is a direct mirror of the upstream transform and as such will be impacted by that transform. This implies that any transforms built upon the Data Feed Connector may become invalid if the schema of the upstream transform changes.
Updated 5 months ago