Custom Docker Images

Introduction

By default, all Ascend Cluster Pools run on our native containers. The containers' image builds upon the latest stable version of Spark and Scala, also bundles up a few useful packages:

• The Ascend SDK
• Ascend's connection definitions
• Ascend's utility functions to make developing easier (such as logging)
• Commonly used Python packages (check out section "Pre-installed Python Packages" below)

When to use custom images

There are two use cases where you might want to change the default image used.

Use your own image with additional Python packages or reusable functions. If you want to include additional Python packages and functions, start with Preparing a Custom Image.

Use a different version of Spark and/or Scala. If you want to use a different version of Spark and/or Scala, continue to Using a Custom Image.