- If you would like to use your own docker image in Ascend, make sure to go through the previous article on "preparing a custom image", to have it properly set up.
First, find Runtime Settings options in a transform:
- In Pyspark and SQL Transforms, it is nested under "Advanced Settings"
- In Scala Transform, it is marked as a required field in the creation flow.
In this dropdown, you'll find the following four options:
Use Data Service Image (default)
This is the default option for Transforms if nothing is specified.
Native Ascend Spark
Once selected, you may choose one of the standard Ascend Spark runtimes.
If you have registered custom labels at the site level, they can be selected here.
Container image URL
Enter an image URL (for example,
To make managing custom images easier in Ascend, you may declare all needed images in the "Site admin" settings and reference them later when building Dataflows.
Only site admins for your environment may operate on this. Reach out to your site admins to help you register needed images.
- Go to the Dashboard from the top left Ascend Icon
- Click on "Admin," then "Cluster Management."
- In the "Custom Docker Images" section, click "Add a new image."
- Give the image an easily referenceable name as a label, for example, "custom-image-with-Arrow-library."
- Enter image URL and required runtime.
- Hit "Create" to finish image registration.
Once complete, you may choose the registered image easily in both Data Service settings and Transforms.
By default, all Transforms in a Data Service runs on the same initial Ascend Spark image. If you would like to default to your custom Docker image, you may:
- Click "Data Service Settings", after hitting the small "Gear" icon in the upper section of the left panel.
- Navigate to "Container Images".
- Choose a default image to be used for this Data Service. Follow the table above to make sense of the options.
- Hit "Update", and all Transforms will default to use the new image.
Register a images from a private image repository
It is possible that your images are not hosted in a public image repository, and may require additional credentials to pull images.
In this case, please reach out to Ascend support (via Intercom or Slack). We will help you set up the configurations so those images can be pulled.
Updated 6 months ago