Python SDK

Ascend Python SDK

Installation & Usage

Requirements: Python 3.8+

pip install

You can install the package directly via pip.

pip install ascend-io-sdk

Then import the package:

import ascend

Getting Started

  1. Follow the installation procedure
  2. Create credentials (access key + secret key) in the Ascend UI.
  3. Instantiate a client by passing in credentials to the Client.
from ascend.sdk.client import Client
from ascend.protos.api.api_pb2 import DataService

client = Client(hostname="trial.ascend.io", access_key="youraccesskey", secret_key="yoursecretkey")
client.create_data_service(DataService(id="demos", name="demos", description="test data service"))

Authorization

Credentials can be passed in to the client directly with the keyword arguments of access_key or secret_key as in the following:

from ascend.sdk.client import Client

client = Client(hostname="trial.ascend.io", access_key="youraccesskey", secret_key="yoursecretkey")

Alternatively, the client can read credentials from a config file, located in ~/.ascend/credentials. The file should have the following structure:

[trial]
ascend_access_key_id=<your_key_id_here>
ascend_secret_access_key=<your_secret_key_here>

where the [trial] should be replaced with your Ascend subdomain, [subdomain].

Then, constructing the client, you do not need to pass in the credentials directly:

from ascend.sdk.client import Client

client = Client(hostname="trial.ascend.io")

Overview

The SDK Python API comes in 2 flavors -

  1. A low-level CRUD API to access, create, update, and delete resources in Ascend.
  2. A high-level declarative API to declare the desired end-state of Dataflows and DataServices.

The use-cases for 1.) and 2.) are different, and complementary to each other.

  • The low-level API is useful for iterating on a Dataflow, and accessing detailed information about components such as schema, records, partitions, lineage, etc. Resources in this API are defined as protobuf objects, and the CRUD operations are available as instance methods on the SDK Client.
  • The high-level API is useful when you want to export and commit your Dataflow or DataService to source-control, or have it be part of a CICD pipeline. Resources in this API are defined as Dataflow resource definition objects. Utility classes such as DataflowApplier or DataServiceApplier are used to apply/sync these definitions with your account. Under the hood, the high-level API is implemented on top of the low-level API.

Getting started with Dataflow resource definitions

There are a couple of ways to get started. You could write a Dataflow from scratch based on the API reference and the examples in this tutorial. However, the typical workflow for a new user is to first create DataServices and Dataflows on the Ascend UI, and then download/export these to their local file-system.

Download an existing Dataflow

To download a Dataflow:

from ascend.sdk.client import Client
from ascend.sdk.render import download_dataflow

client = Client("myhost.ascend.io")
download_dataflow(client,
                  data_service_id="my_data_service",
                  dataflow_id="my_dataflow",
                  resource_base_path="./")

This generates a Dataflow definition in a Python file named my_dataflow.py, and stores inline code from Transforms and ReadConnectors as separate files in a sub-folder named my_dataflow/ under resource_base_path.

Feel free to inspect and refactor the generated files. You'll notice that the Dataflow definition consists roughly of:

  1. A series of import statements to pull in the required SDK and protobuf classes.
  2. A Dataflow resource definition that includes component definitions, data feeds, data feed connectors, and component groups. Note how relationships between resources are expressed. For instance, dependencies between components are expressed via fields such as input_id (for WriteConnectors) and input_ids (for Transforms).
  3. Creation of a SDK Client instance.
  4. Create a DataflowApplier and apply the constructed Dataflow definition.
    Once downloaded, this Python script can be run any number of times to achieve the desired end-state for the Dataflow.

The steps for downloading a DataService are similar:

from ascend.sdk.client import Client
from ascend.sdk.render import download_data_service

client = Client("myhost.ascend.io")
download_data_service(client,
                      data_service_id="my_data_service",
                      resource_base_path="./")

This generates a DataService definition in a Python file my_data_service.py under resource_base_path, creates a sub-folder for each Dataflow, and stores inline code for Transforms and ReadConnectors in these sub-folders.

Getting started with Protobuf classes

The Ascend model classes and types are all generated from Google's protobufs. For developers that are used to protobufs, these classes should feel familiar. For developers new to Protobuf classes, however, this section provides a brief overview.

Constructing an Object

Attributes to an object can be passed into the constructor with keyword syntax, like the following:

from ascend.protos.api.api_pb2 import DataService

ds = DataService(id="demos", name="demos", description="test data service")

Alternatively, fields can be set directly through assignment:

from ascend.protos.api.api_pb2 import DataService

ds = DataService()
ds.id = "demos"
ds.name = "demos"
ds.description = "test data service"

Nested Types

Fields for a nested type can be nested in the constructor, as so:

from ascend.protos.external.external_pb2 import Aws, Credentials

b = Aws.S3.Bucket(name="mybucket", credential_id=Credentials.Id(value="id-3"))

However, it is not allowed to assign the nested field directly:

b = Aws.S3.Bucket()
b.credential_id = Credentials.Id(value="id-3") # WRONG!

In this example, to set credential_id, either set the value directly or use CopyFrom:

b = Aws.S3.Bucket()
b.credential_id.value = "id-3"
b.credential_id.CopyFrom(Credentials.Id(value="id-3"))

For more information on setting nested types, please see the Protobuf Python reference on embedded messages.

Working with Lists

Working with lists is similar to working with nested types, in that the list is already instantiated.

from ascend.protos.api.api_pb2 import Transform

t = Transform()
t.inputs.append(Transform.Input(uuid="fake-uuid")) # Append one value
t.inputs.extend([Transform.Input(uuid="fake-uuid2"), Transform.Input(uuid="fake-uuid3")]) # Append a whole list
len(t.inputs) # returns 3

For more information on working with lists, please see the Protobuf Python reference on repeated fields and repeated message fields.

Ascend SDK API Reference

In addition to the SDK Client, the SDK distribution contains 4 top-level modules for interacting with Ascend.

Module definitions

Ascend resource definition classes.

Classes

Component(id: str)

Components are the functional units in a Dataflow.
There are three main types of Components: ReadConnectors, Transforms, and WriteConnectors.

ComponentGroup(id: str, name: str, component_ids: List[str], description: str = None)

A component group

Create a ComponentGroup definition.

Parameters:

  • id: identifier that is unique within Dataflow
  • name: user-friendly name
  • description: brief description
  • component_ids: list of ids of components that make up the group
    Returns: a new ComponentGroup instance.
DataFeed(id: str, name: str, input_id: str, description: str = None, shared_with_all: bool = False, data_services_shared_with: List[str] = [], hidden_from_host_data_service: bool = False)

A data feed

Create a DataFeed definition.

Parameters:

  • id: identifier that is unique within Dataflow
  • name: user-friendly name
  • description: brief description
  • input_id: input component id
  • shared_with_all: if set to true, this data feed is shared with all data services,
    including the host data service
  • data_services_shared_with: a list of ids of data services that we want to share
    data with (not including the host data service)
  • hidden_from_host_data_service: if set to true, the host data service cannot subscribe
    to the contents of this data feed
    Returns: a new DataFeed instance.

Note - if shared_with_all is set to true, the values of data_services_shared_with and
hidden_from_host_data_service are ignored

DataFeedConnector(id: str, name: str, input_data_service_id: str, input_dataflow_id: str, input_data_feed_id: str, description: str = None)

A data feed connector

Create a DataFeedConnector definition.

Parameters:

  • id: identifier that is unique within Dataflow
  • name: user-friendly name
  • description: brief description
  • input_data_service_id: id of the DataService that the input DataFeed belongs to
  • input_dataflow_id: id of the Dataflow that the input DataFeed belongs to
  • input_data_feed_id: input DataFeed id
    Returns: a new DataFeedConnector instance.
DataService(id: str, name: str, description: str = None, dataflows: List[definitions.Dataflow] = [])

A DataService is the highest-level organizational structure in Ascend, and the primary
means of controlling access. It contains one or more Dataflows and can communicate with
other Data Services via Data Feeds.

Create a DataService definition.

Parameters:

  • id: DataService identifier
  • name: name
  • description: brief description
  • dataflows: list of dataflows
    Returns: a new DataService instance.
Dataflow(id: str, name: str, description: str = None, components: List[definitions.Component] = [], data_feed_connectors: List[definitions.DataFeedConnector] = [], data_feeds: List[definitions.DataFeed] = [], groups: List[definitions.ComponentGroup] = [])

A Dataflow is a representation of the operations being performed on data, expressed
as a dependency graph. Operations are expressed declaratively via components, and are
broadly categorized as read connectors (data ingestion), transforms (transformation),
and write connectors. Data feeds expose the results of transforms to other dataflows
and data services. Data feed connectors are subscriptions to data feeds. Component groups
offer users a convenient way to organize components that have a similar purpose.

Create a Dataflow definition.

Parameters

  • id: identifier for the Dataflow that is unique within its DataService
  • name: user-friendly name
  • description: brief description
  • components: list of components
  • data_feed_connectors: list of data feeds
  • data_feeds: list of data feed connectors
  • groups: list of component groups
    Returns: a new Dataflow instance.
Definition()

Base class for all definitions.

ReadConnector(id: str, name: str, container: ascend.protos.io.io_pb2.Container, description: str = None, pattern: ascend.protos.pattern.pattern_pb2.Pattern = None, update_periodical: ascend.protos.core.core_pb2.Periodical = None, assigned_priority: ascend.protos.component.component_pb2.Priority = None, last_manual_refresh_time: google.protobuf.timestamp_pb2.Timestamp = None, aggregation_limit: google.protobuf.wrappers_pb2.Int64Value = None, bytes: ascend.protos.component.component_pb2.FromBytes = None, records: ascend.protos.component.component_pb2.FromRecords = None)

A read connector

Create a ReadConnector definition.

Parameters:

  • id: identifier that is unique within Dataflow
  • name: user-friendly name
  • description: brief description
  • container (proto): a typed representation of the source of data
  • pattern (proto): defines the set of objects in the source that will be ingested
  • update_periodical: specifies source update frequency
  • assigned_priority: scheduling priority for the read connector
  • last_manual_refresh_time:
  • aggregation_limit:
  • bytes: defines the operator for parsing bytes read from the source (example - json
    or xsv parsing)
  • records: defines the schema for records read from a database or warehouse
    Returns: a new ReadConnector instance.

Note - a read connector can have either bytes or records defined, but not both.

Transform(id: str, name: str, input_ids: List[str], operator: ascend.protos.operator.operator_pb2.Operator, description: str = None, assigned_priority: ascend.protos.component.component_pb2.Priority = None)

A transform

Create a Transform definition.

Parameters:

  • id: identifier that is unique within Dataflow
  • name: user-friendly name
  • description: brief description
  • input_ids: list of input component ids
  • operator: transform operator definition - example - SQL/PySpark/Scala transform
  • assigned_priority: scheduling priority for transform
    Returns: a new Transform instance.
WriteConnector(id: str, name: str, input_id: str, container: ascend.protos.io.io_pb2.Container, description: str = None, assigned_priority: ascend.protos.component.component_pb2.Priority = None, bytes: ascend.protos.component.component_pb2.ToBytes = None, records: ascend.protos.component.component_pb2.ToRecords = None)

A write connector

Create a WriteConnector definition.

Parameters:

  • id: identifier that is unique within Dataflow
  • name: user-friendly name
  • description: brief description
  • input_id: input component id
  • container (proto): a typed representation of the destination of data
  • assigned_priority: scheduling priority of write connector
  • bytes: defines the operator for formatting bytes written to the object store destination
  • records: indicates that the destination is a structured record store like a database
    or a data warehouse
    Returns: a new WriteConnector instance.

Note - a write connector can have either bytes or records defined, but not both.

Module applier

Ascend resource applier classes - used for applying resource definitions.

Classes

DataServiceApplier(client: ascend.sdk.client.Client)

DataServiceApplier is a utility class that accepts a DataService definition and
'applies' it - ensuring that the DataService is created if it does not already exist,
deleting any existing Dataflows, components, and members of the DataService that
are not part of the supplied definition, and applying any configuration changes needed
to match the definition.

Creates a new DataServiceApplier.

Methods
apply(self, data_service: ascend.sdk.definitions.DataService, delete=True, dry_run=False)

Create or update the specified DataService.

Parameters:

  • data_service: DataService definition
  • delete (optional): If set to True (which is the default) - delete any Dataflow
    that is not part of data_service. At the Dataflow level, remove any components
    that are not part of the Dataflow definition.
  • dry_run (optional): If set to True (False by default) - skips any create or
    update operations
DataflowApplier(client: ascend.sdk.client.Client)

DataflowApplier is a utility class that accepts a Dataflow definition and 'applies'
it - ensuring that a Dataflow is created if it does not already exist and binding it to
the DataService identified by data_service_id, deleting any components and members of
the Dataflow that are not part of the supplied definition, and applying any configuration
changes needed to match the definition.

Creates a new DataflowApplier.

Methods
apply(self, data_service_id: str, dataflow: ascend.sdk.definitions.Dataflow, delete=True, dry_run=False)

Accepts a Dataflow definition, and ensures that it is created, bound to the DataService
identified by data_service_id, and has all of the constituent elements included in the
Dataflow definition - components (read connectors, transforms, write connectors), component
groups, data feeds, and data feed connectors. The specified DataService must already exist.

Parameters:

  • data_service_id: DataService id
  • dataflow: Dataflow definition
  • delete: if set to True (default=True) - delete any components, data feeds,
    data feed connectors, or groups not defined in dataflow
  • dry_run: If set to True (default=False) - skips any create or update operations
ComponentApplier.build(client: ascend.sdk.client.Client, data_service_id: str, dataflow_id: str)

ComponentApplier is a utility class that accepts a Component definition (ReadConnector, WriteConnector, Transform) and 'applies'
it - ensuring that the Component is created if it does not already exist and binding it to
the DataService identified by data_service_id and the DataFlow identified by
dataflow_id, crating or applying any configuration changes needed to match the definition.

Creates a new ComponentApplier that has the apply function.

Methods
apply(self, data_service_id: str, dataflow_id: str, component: Component)

Accepts a Component definition, and ensures that it is created, bound to the DataService
identified by data_service_id and the Dataflow identified by dataflow_id, and has
all of the properties in the Component definition The specified DataService and Dataflow must already exist.

Parameters:

  • data_service_id: DataService id
  • dataflow_id: Dataflow id
  • component: Component Definition (one of ReadConnector, Transform, WriteConnector)

Module render

Utility functions to download Ascend resource definitions.

Functions

from ascend.sdk.render import download_data_service

download_data_service(client: ascend.sdk.client.Client, data_service_id: str, resource_base_path: str = '.')

Downloads a DataService and writes its definition to a file named {data_service_id}.py
under resource_base_path. Inline code for PySpark and SQL Transforms are written as separate
files to sub-folders - resource_base_path/{dataflow_id}/ with the file name derived from
the id of the component to which the code belongs. Raises ValueError if resource_base_path
does not exist.

Parameters:

  • client: SDK client
  • data_service_id: DataService id
  • resource_base_path: base path to which DataService definition and resource
    files are written
from ascend.sdk.render import download_dataflow

download_dataflow(client: ascend.sdk.client.Client, data_service_id: str, dataflow_id: str, resource_base_path: str = '.')

Downloads a Dataflow and writes its definition to a file named {dataflow_id}.py under
resource_base_path. Inline code for PySpark and SQL Transforms are written as separate
files to a sub-folder - resource_base_path/{dataflow_id}/ with the file names derived
from the id of the component to which the code belongs. Raises a ValueError if
resource_base_path does not exist.

Parameters:

  • client: SDK client
  • data_service_id: DataService id
  • dataflow_id: Dataflow id
  • resource_base_path: path to which Dataflow definition and resource files
    are written
from ascend.sdk.render import download_component

download_component(client: ascend.sdk.client.Client, data_service_id: str, dataflow_id: str, component_id: str, resource_base_path: str = '.')

Downloads a Component and writes its definition to a file named {component_id}.py under
resource_base_path. Inline code for PySpark and SQL Transforms are written as separate
files in the same folder with the file names derived from the id of the component to
which the code belongs. Raises a ValueError if resource_base_path does not exist.

Parameters:

  • client: SDK client
  • data_service_id: DataService id
  • dataflow_id: Dataflow id
  • component_id: Component id
  • resource_base_path: path to which Component definition and resource files are written

Classes

InlineCode(code: str, attribute_path: Tuple[str, ...], resource_path: str, base_path: str, base64_encoded: bool = False)

Represents the result of extraction of inline code from a component - such as
a sql statement or PySpark code. Contains all of the metadata needed to render
a component definition, with the inline code written to a separate file, and a reference
to this code stored in the component.

Parameters:

  • code: inline code
  • attribute_path: path of the attribute in the component definition that contains
    the inline code, represented as a tuple of path components. For instance, the
    path for sql code in a Transform is ("operator", "sql_query", "sql")
  • resource_path: file path to which inline code is written
  • base_path: base path of dataflow or data service resource definition
  • base64_encoded: if set to True, inline code is base64 encoded

Module builder

Utility functions to create Ascend resource definitions from the corresponding protobuf
objects. The protobuf objects can be obtained via the 'imperative' API accessible through
the SDK client -
client.get_data_service(<data_service_id>)
client.get_dataflow(<dataflow_id>)

Functions

data_service_from_proto(client: ascend.sdk.client.Client, proto: ascend.protos.api.api_pb2.DataService) ‑> ascend.sdk.definitions.DataService

Build a DataService definition from the API DataService protobuf representation.
This is a utility function that helps bridge the declarative and imperative forms
of the SDK by transforming a DataService proto message to a DataService definition
that can be 'applied'.

Parameters:

  • client: SDK client
  • proto: DataService protobuf object
    Returns: a DataService definition
dataflow_from_proto(client: ascend.sdk.client.Client, data_service_id: str, proto: ascend.protos.api.api_pb2.Dataflow) ‑> ascend.sdk.definitions.Dataflow

Build a Dataflow definition from the API Dataflow protobuf representation.

Parameters:

  • client: SDK client
  • data_service_id: DataService id
  • proto: Dataflow protobuf object
    Returns: a Dataflow definition

Ascend Client method summary

MethodDescriptionMatching HTTP Request
create_component_groupcreate component group in dataflowPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/groups
create_connectioncreate connectionPOST /api/v1/organizations/{data_service_id}/connections
create_container_imagecreate a container image definitionPOST /api/v1/envs/container_images
create_data_feedcreate data feed in dataflowPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs
create_data_feed_connectorcreate a data feed connector in dataflowPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs
create_dataflowcreate dataflowPOST /api/v1/organizations/{data_service_id}/projects
create_data_servicecreate data servicePOST /api/v1/organizations
create_data_service_credentialscreate data service credentialsPOST /api/v1/organizations/{data_service_id}/vault
create_read_connectorcreate read connector in dataflowPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources
create_transformcreate transform in dataflowPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views
create_write_connectorcreate write connector in dataflowPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks
delete_component_groupdelete component group in dataflowDELETE /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/groups/{id}
delete_connectiondelete connectionDELETE /api/v1/organizations/{data_service_id}/connections/{connection_id}
delete_container_imagedelete container image definition by labelDELETE /api/v1/envs/container_images/{label}
delete_data_feeddelete data feed in dataflowDELETE /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs/{id}
delete_data_feed_connectordelete data feed connector in dataflowDELETE /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{id}
delete_dataflowdelete dataflowDELETE /api/v1/organizations/{data_service_id}/projects/{dataflow_id}
delete_data_servicedelete data serviceDELETE /api/v1/organizations/{data_service_id}
delete_data_service_credentialsdelete credentials by idDELETE /api/v1/organizations/{data_service_id}/vault/{credential_id}
delete_read_connectordelete read connector in dataflowDELETE /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}
delete_transformdelete transform in dataflowDELETE /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}
delete_write_connectordelete write connector in dataflowDELETE /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}
download_logsdownload log archive for componentGET /api/v1/logs/download
get_component_groupget component group in dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/groups/{id}
get_connectionget connectionGET /api/v1/organizations/{data_service_id}/connections/{connection_id}
get_connection_typeget connection typeGET /api/v1/organizations/{data_service_id}/connection_types/{connection_type_id}
get_container_imageget container image definition by labelGET /api/v1/envs/container_images/{label}
get_data_feedget data feed in dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs/{id}
get_data_feed_connectorget data feed connector in dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{id}
get_data_feed_connector_partitionsget data feed connector partitionsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{id}/partitions
get_data_feed_connector_recordsget data feed connector recordsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{id}/records
get_data_feed_connector_schemaget data feed connector schemaGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{id}/schema
get_data_feed_connector_stateget data feed connector stateGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{id}/state
get_data_feed_connector_statisticsget data feed connector statisticsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{id}/statistics
get_data_feed_for_data_feed_connectorget input data feed for a data feed connectorGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{id}/pub
get_data_feed_partitionsget data feed partitionsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs/{id}/partitions
get_data_feed_recordsget data feed recordsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs/{id}/records
get_data_feed_schemaget data feed schemaGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs/{id}/schema
get_data_feed_stateget data feed stateGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs/{id}/state
get_data_feed_statisticsget data feed statisticsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs/{id}/statistics
get_dataflowget dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}
get_data_serviceget data serviceGET /api/v1/organizations/{data_service_id}
get_data_service_runtime_environmentget runtime environment of a data serviceGET /api/v1/organizations/{data_service_id}/runtime_environment
get_logsget logs for componentGET /api/v1/logs
get_read_connectorget read connector in dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}
get_read_connector_errorsget read connector errorsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/errors
get_read_connector_lineageget read connector lineageGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/lineage
get_read_connector_partitionsget read connector partitionsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/partitions
get_read_connector_recordsget read connector recordsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/records
get_read_connector_schemaget read connector schemaGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/schema
get_read_connector_stateget read connector stateGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/state
get_read_connector_statisticsget read connector statisticsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/statistics
get_read_connector_warningsget read connector warningsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/warnings
get_transformget transform in dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}
get_transform_errorsget transform errorsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/errors
get_transform_lineageget transform lineageGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/lineage
get_transform_partitionsget transform partitionsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/partitions
get_transform_recordsget transform recordsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/records
get_transform_schemaget transform schemaGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/schema
get_transform_stateget transform stateGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/state
get_transform_statisticsget transform statisticsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/statistics
get_transform_warningsget transform warningsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/warnings
get_write_connectorget write connector in dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}
get_write_connector_errorsget write connector errorsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}/errors
get_write_connector_lineageget write connector lineageGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}/lineage
get_write_connector_partitionsget write connector partitionsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}/partitions
get_write_connector_stateget write connector stateGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}/state
get_write_connector_statisticsget write connector statisticsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}/statistics
get_write_connector_warningsget write connector warningsGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}/warnings
list_component_groupslist component groups in a dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/groups
list_connectionsList all connections in a data serviceGET /api/v1/organizations/{data_service_id}/connections
list_connection_typesList all connection types in a data serviceGET /api/v1/organizations/{data_service_id}/connection_types
list_container_imageslist container image definitionsGET /api/v1/envs/container_images
list_data_feed_connectorslist data feed connectors in a dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs
list_data_feedslist data feeds in a dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs
list_dataflow_componentslist components in a dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/components
list_dataflowsList all dataflows in a data serviceGET /api/v1/organizations/{data_service_id}/projects
list_data_service_componentsget list of components in a data serviceGET /api/v1/organizations/{data_service_id}/components
list_data_service_credentialslist data service credentialsGET /api/v1/organizations/{data_service_id}/vault
list_data_service_rolesList all Data Service rolesGET /authz/v1/org_roles
list_data_servicesList all Data ServicesGET /api/v1/organizations
list_read_connectorslist read connectors in a dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources
list_transformslist transforms in a dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views
list_write_connectorslist write connectors in a dataflowGET /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks
refresh_read_connectorrefresh read connectorPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/refresh
reset_dataflow_errorsreset dataflow errorsPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/reset-errors
reset_read_connector_errorsreset read connector errorsPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/reset-errors
reset_transform_errorsreset transform errorsPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/reset-errors
reset_write_connector_errorsreset write connector errorsPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}/reset-errors
share_connectionshare connection to another data servicePOST /api/v1/organizations/{data_service_id}/connections/{connection_id}/share_to/{target_data_service_id}
update_component_groupupdate component group in dataflowPATCH /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/groups/{component_group.id}
update_connectionupdate connectionPATCH /api/v1/organizations/{data_service_id}/connections/{connection.id.value}
update_container_imageupdate a container image definitionPATCH /api/v1/envs/container_images/{label}
update_data_feedupdate data feed in dataflowPATCH /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/pubs/{data_feed.id}
update_data_feed_connectorupdate data feed connector in dataflowPATCH /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/subs/{data_feed_connector.id}
update_dataflowupdate dataflowPATCH /api/v1/organizations/{data_service_id}/projects/{dataflow.id}
update_data_serviceupdate data servicePATCH /api/v1/organizations/{id}
update_data_service_credentialsupdate data service credentialsPATCH /api/v1/organizations/{data_service_id}/vault/{credential_id}
update_data_service_runtime_environmentupdate runtime environment of a data servicePATCH /api/v1/organizations/{data_service_id}/runtime_environment
update_read_connectorupdate read connector in dataflowPATCH /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{read_connector.id}
update_read_connector_paused_statepause or unpause a read connectorPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sources/{id}/paused-state
update_transformupdate transform in dataflowPATCH /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{transform.id}
update_transform_paused_statepause or unpause a transformPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/views/{id}/paused-state
update_write_connectorupdate write connector in dataflowPATCH /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{write_connector.id}
update_write_connector_paused_statepause or unpause a write connectorPOST /api/v1/organizations/{data_service_id}/projects/{dataflow_id}/sinks/{id}/paused-state

Client Methods Breakdown

create_component_group

create component group in dataflow

Returns: ComponentGroupResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
bodyComponentGroupComponent group details.True

[Back to top] [Back to API list] [Back to Models]

create_connection

create connection

Returns: CreateConnectionResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringTrue
bodyConnectionTrue

[Back to top] [Back to API list] [Back to Models]

create_container_image

create a container image definition

Returns: ContainerImageResponse

Parameters
NameTypeDescriptionRequired
bodySpark.ContainerImageTrue

[Back to top] [Back to API list] [Back to Models]

create_data_feed

create data feed in dataflow

Returns: DataFeedResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
bodyDataFeedData feed details.True

[Back to top] [Back to API list] [Back to Models]

create_data_feed_connector

create a data feed connector in dataflow

Returns: DataFeedConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
bodyDataFeedConnectorData feed connector details.True

[Back to top] [Back to API list] [Back to Models]

create_dataflow

create dataflow

Returns: DataflowResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
bodyDataflowDataflow details.True

[Back to top] [Back to API list] [Back to Models]

create_data_service

create data service

Returns: DataServiceResponse

Parameters
NameTypeDescriptionRequired
bodyDataServiceTrue

[Back to top] [Back to API list] [Back to Models]

create_data_service_credentials

create data service credentials

Returns: CredentialsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
bodyCredentialsComponent credentials.True

[Back to top] [Back to API list] [Back to Models]

create_read_connector

create read connector in dataflow

Returns: ReadConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
bodyReadConnectorRead connector to be created/updated.True

[Back to top] [Back to API list] [Back to Models]

create_transform

create transform in dataflow

Returns: TransformResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
bodyTransformTransform details.True

[Back to top] [Back to API list] [Back to Models]

create_write_connector

create write connector in dataflow

Returns: WriteConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
bodyWriteConnectorWrite connector details.True

[Back to top] [Back to API list] [Back to Models]

delete_component_group

delete component group in dataflow

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

delete_connection

delete connection

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringTrue
connection_idstringTrue

[Back to top] [Back to API list] [Back to Models]

delete_container_image

delete container image definition by label

Returns: ContainerImageDeleteResponse

Parameters
NameTypeDescriptionRequired
labelstringcontainer label.True

[Back to top] [Back to API list] [Back to Models]

delete_data_feed

delete data feed in dataflow

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

delete_data_feed_connector

delete data feed connector in dataflow

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

delete_dataflow

delete dataflow

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True

[Back to top] [Back to API list] [Back to Models]

delete_data_service

delete data service

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True

[Back to top] [Back to API list] [Back to Models]

delete_data_service_credentials

delete credentials by id

Returns: DeleteCredentialsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
credential_idstringCredential id.True

[Back to top] [Back to API list] [Back to Models]

delete_read_connector

delete read connector in dataflow

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

delete_transform

delete transform in dataflow

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

delete_write_connector

delete write connector in dataflow

Returns: DeleteResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

download_logs

download log archive for component

Returns: object

Parameters
NameTypeDescriptionRequired
range_startint64start time in nanoseconds.False
range_endint64end time in nanoseconds.False
labelstringlog label.False
backwardbooleanlog search direction, set to true to get most recent logs first.False
jwtstringjwt for validation.False
limitintegermax number of lines in result.False
next_page_cursorint64cursor to get next page.False

[Back to top] [Back to API list] [Back to Models]

get_component_group

get component group in dataflow

Returns: ComponentGroupResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_connection

get connection

Returns: ConnectionResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringTrue
connection_idstringTrue

[Back to top] [Back to API list] [Back to Models]

get_connection_type

get connection type

Returns: ConnectionTypeResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringTrue
connection_type_idstringTrue

[Back to top] [Back to API list] [Back to Models]

get_container_image

get container image definition by label

Returns: ContainerImageResponse

Parameters
NameTypeDescriptionRequired
labelstringcontainer label.True

[Back to top] [Back to API list] [Back to Models]

get_data_feed

get data feed in dataflow

Returns: DataFeedResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_data_feed_connector

get data feed connector in dataflow

Returns: DataFeedConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_data_feed_connector_partitions

get data feed connector partitions

Returns: ComponentPartitionsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetintegerOffset in partition metadata list.False
limitintegerUpper limit of partition metadata results returned.False
start_tsint64False
end_tsint64False

[Back to top] [Back to API list] [Back to Models]

get_data_feed_connector_records

get data feed connector records

Returns: ComponentRecordsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetuint64Record offset.False
limitint64Upper limit of number of records to be returned.False
tokenstringFalse
formatstringFalse

[Back to top] [Back to API list] [Back to Models]

get_data_feed_connector_schema

get data feed connector schema

Returns: ComponentSchemaResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_data_feed_connector_state

get data feed connector state

Returns: ComponentStateResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_data_feed_connector_statistics

get data feed connector statistics

Returns: ComponentStatisticsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse

[Back to top] [Back to API list] [Back to Models]

get_data_feed_for_data_feed_connector

get input data feed for a data feed connector

Returns: DataFeedResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_data_feed_partitions

get data feed partitions

Returns: ComponentPartitionsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetintegerOffset in partition metadata list.False
limitintegerUpper limit of partition metadata results returned.False
start_tsint64False
end_tsint64False

[Back to top] [Back to API list] [Back to Models]

get_data_feed_records

get data feed records

Returns: ComponentRecordsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetuint64Record offset.False
limitint64Upper limit of number of records to be returned.False
tokenstringFalse
formatstringFalse

[Back to top] [Back to API list] [Back to Models]

get_data_feed_schema

get data feed schema

Returns: ComponentSchemaResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_data_feed_state

get data feed state

Returns: ComponentStateResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_data_feed_statistics

get data feed statistics

Returns: ComponentStatisticsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse

[Back to top] [Back to API list] [Back to Models]

get_dataflow

get dataflow

Returns: DataflowResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True

[Back to top] [Back to API list] [Back to Models]

get_data_service

get data service

Returns: DataServiceResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True

[Back to top] [Back to API list] [Back to Models]

get_data_service_runtime_environment

get runtime environment of a data service

Returns: DataServiceRuntimeEnvironmentResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True

[Back to top] [Back to API list] [Back to Models]

get_logs

get logs for component

Returns: ComponentLogsResponse

Parameters
NameTypeDescriptionRequired
range_startint64start time in nanoseconds.False
range_endint64end time in nanoseconds.False
labelstringlog label.False
backwardbooleanlog search direction, set to true to get most recent logs first.False
jwtstringjwt for validation.False
limitintegermax number of lines in result.False
next_page_cursorint64cursor to get next page.False

[Back to top] [Back to API list] [Back to Models]

get_read_connector

get read connector in dataflow

Returns: ReadConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_read_connector_errors

get read connector errors

Returns: ComponentErrorsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
offsetuint64Offset in errors list.False
limituint64Upper limit of the number of errors returned.False

[Back to top] [Back to API list] [Back to Models]

get_read_connector_lineage

get read connector lineage

Returns: ComponentLineageResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_read_connector_partitions

get read connector partitions

Returns: ComponentPartitionsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetintegerOffset in partition metadata list.False
limitintegerUpper limit of partition metadata results returned.False
start_tsint64False
end_tsint64False

[Back to top] [Back to API list] [Back to Models]

get_read_connector_records

get read connector records

Returns: ComponentRecordsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetuint64Record offset.False
limitint64Upper limit of number of records to be returned.False
tokenstringFalse
formatstringFalse

[Back to top] [Back to API list] [Back to Models]

get_read_connector_schema

get read connector schema

Returns: ComponentSchemaResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_read_connector_state

get read connector state

Returns: ComponentStateResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_read_connector_statistics

get read connector statistics

Returns: ComponentStatisticsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse

[Back to top] [Back to API list] [Back to Models]

get_read_connector_warnings

get read connector warnings

Returns: ComponentWarningsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
offsetuint64Offset in errors list.False
limituint64Upper limit of the number of errors returned.False

[Back to top] [Back to API list] [Back to Models]

get_transform

get transform in dataflow

Returns: TransformResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_transform_errors

get transform errors

Returns: ComponentErrorsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
offsetuint64Offset in errors list.False
limituint64Upper limit of the number of errors returned.False

[Back to top] [Back to API list] [Back to Models]

get_transform_lineage

get transform lineage

Returns: ComponentLineageResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_transform_partitions

get transform partitions

Returns: ComponentPartitionsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetintegerOffset in partition metadata list.False
limitintegerUpper limit of partition metadata results returned.False
start_tsint64False
end_tsint64False

[Back to top] [Back to API list] [Back to Models]

get_transform_records

get transform records

Returns: ComponentRecordsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetuint64Record offset.False
limitint64Upper limit of number of records to be returned.False
tokenstringFalse
formatstringFalse

[Back to top] [Back to API list] [Back to Models]

get_transform_schema

get transform schema

Returns: ComponentSchemaResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_transform_state

get transform state

Returns: ComponentStateResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_transform_statistics

get transform statistics

Returns: ComponentStatisticsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse

[Back to top] [Back to API list] [Back to Models]

get_transform_warnings

get transform warnings

Returns: ComponentWarningsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
offsetuint64Offset in errors list.False
limituint64Upper limit of the number of errors returned.False

[Back to top] [Back to API list] [Back to Models]

get_write_connector

get write connector in dataflow

Returns: WriteConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_write_connector_errors

get write connector errors

Returns: ComponentErrorsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
offsetuint64Offset in errors list.False
limituint64Upper limit of the number of errors returned.False

[Back to top] [Back to API list] [Back to Models]

get_write_connector_lineage

get write connector lineage

Returns: ComponentLineageResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_write_connector_partitions

get write connector partitions

Returns: ComponentPartitionsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse
offsetintegerOffset in partition metadata list.False
limitintegerUpper limit of partition metadata results returned.False
start_tsint64False
end_tsint64False

[Back to top] [Back to API list] [Back to Models]

get_write_connector_state

get write connector state

Returns: ComponentStateResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

get_write_connector_statistics

get write connector statistics

Returns: ComponentStatisticsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
stagestringFalse
stage_indexintegerFalse

[Back to top] [Back to API list] [Back to Models]

get_write_connector_warnings

get write connector warnings

Returns: ComponentWarningsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
offsetuint64Offset in errors list.False
limituint64Upper limit of the number of errors returned.False

[Back to top] [Back to API list] [Back to Models]

list_component_groups

list component groups in a dataflow

Returns: ComponentGroupsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True

[Back to top] [Back to API list] [Back to Models]

list_connections

List all connections in a data service

Returns: ConnectionsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True

[Back to top] [Back to API list] [Back to Models]

list_connection_types

List all connection types in a data service

Returns: ConnectionTypesResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True

[Back to top] [Back to API list] [Back to Models]

list_container_images

list container image definitions

Returns: ContainerImagesResponse

Parameters

This method does not need any parameters.

[Back to top] [Back to API list] [Back to Models]

list_data_feed_connectors

list data feed connectors in a dataflow

Returns: DataFeedConnectorsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
deepbooleanIf set to true, populate the detailed source/view/sink fields in Component response,
and not just top level Component fields.False
statestringComma-separated list of component states to filter on - valid values are
uptodate, running, outofdate, error.False
kindstringComma-separated list of component types to filter on - valid values are

source, view, sink, pub, sub, group. Note - these are legacy names for
ReadConnector, Transform, WriteConnector, DataFeed, DataFeedConnector,
ComponentGroup respectively. | False
summary | boolean | | False

[Back to top] [Back to API list] [Back to Models]

list_data_feeds

list data feeds in a dataflow

Returns: DataFeedsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
deepbooleanIf set to true, populate the detailed source/view/sink fields in Component response,
and not just top level Component fields.False
statestringComma-separated list of component states to filter on - valid values are
uptodate, running, outofdate, error.False
kindstringComma-separated list of component types to filter on - valid values are

source, view, sink, pub, sub, group. Note - these are legacy names for
ReadConnector, Transform, WriteConnector, DataFeed, DataFeedConnector,
ComponentGroup respectively. | False
summary | boolean | | False

[Back to top] [Back to API list] [Back to Models]

list_dataflow_components

list components in a dataflow

Returns: ComponentsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
deepbooleanIf set to true, populate the detailed source/view/sink fields in Component response,
and not just top level Component fields.False
statestringComma-separated list of component states to filter on - valid values are
uptodate, running, outofdate, error.False
kindstringComma-separated list of component types to filter on - valid values are

source, view, sink, pub, sub, group. Note - these are legacy names for
ReadConnector, Transform, WriteConnector, DataFeed, DataFeedConnector,
ComponentGroup respectively. | False
summary | boolean | | False

[Back to top] [Back to API list] [Back to Models]

list_dataflows

List all dataflows in a data service

Returns: DataflowsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True

[Back to top] [Back to API list] [Back to Models]

list_data_service_components

get list of components in a data service

Returns: ComponentsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
deepbooleanIf set to true, populate the detailed source/view/sink fields in Component response,
and not just top level Component fields.False
statestringComma-separated list of component states to filter on - valid values are
uptodate, running, outofdate, error.False
kindstringComma-separated list of component types to filter on - valid values are

source, view, sink, pub, sub, group. Note - these are legacy names for
ReadConnector, Transform, WriteConnector, DataFeed, DataFeedConnector,
ComponentGroup respectively. | False
summary | boolean | | False

[Back to top] [Back to API list] [Back to Models]

list_data_service_credentials

list data service credentials

Returns: ListCredentialsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
component_hintstringFalse

[Back to top] [Back to API list] [Back to Models]

list_data_service_roles

List all Data Service roles

Returns: DataServiceRolesResponse

Parameters

This method does not need any parameters.

[Back to top] [Back to API list] [Back to Models]

list_data_services

List all Data Services

Returns: DataServicesResponse

Parameters

This method does not need any parameters.

[Back to top] [Back to API list] [Back to Models]

list_read_connectors

list read connectors in a dataflow

Returns: ReadConnectorsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
deepbooleanIf set to true, populate the detailed source/view/sink fields in Component response,
and not just top level Component fields.False
statestringComma-separated list of component states to filter on - valid values are
uptodate, running, outofdate, error.False
kindstringComma-separated list of component types to filter on - valid values are

source, view, sink, pub, sub, group. Note - these are legacy names for
ReadConnector, Transform, WriteConnector, DataFeed, DataFeedConnector,
ComponentGroup respectively. | False
summary | boolean | | False

[Back to top] [Back to API list] [Back to Models]

list_transforms

list transforms in a dataflow

Returns: TransformsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
deepbooleanIf set to true, populate the detailed source/view/sink fields in Component response,
and not just top level Component fields.False
statestringComma-separated list of component states to filter on - valid values are
uptodate, running, outofdate, error.False
kindstringComma-separated list of component types to filter on - valid values are

source, view, sink, pub, sub, group. Note - these are legacy names for
ReadConnector, Transform, WriteConnector, DataFeed, DataFeedConnector,
ComponentGroup respectively. | False
summary | boolean | | False

[Back to top] [Back to API list] [Back to Models]

list_write_connectors

list write connectors in a dataflow

Returns: WriteConnectorsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
deepbooleanIf set to true, populate the detailed source/view/sink fields in Component response,
and not just top level Component fields.False
statestringComma-separated list of component states to filter on - valid values are
uptodate, running, outofdate, error.False
kindstringComma-separated list of component types to filter on - valid values are

source, view, sink, pub, sub, group. Note - these are legacy names for
ReadConnector, Transform, WriteConnector, DataFeed, DataFeedConnector,
ComponentGroup respectively. | False
summary | boolean | | False

[Back to top] [Back to API list] [Back to Models]

refresh_read_connector

refresh read connector

Returns: ReadConnectorRefreshResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True

[Back to top] [Back to API list] [Back to Models]

reset_dataflow_errors

reset dataflow errors

Returns: DataflowErrorsResetResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
bodyDataflowErrorsResetRequest.OptionsReset request options.True

[Back to top] [Back to API list] [Back to Models]

reset_read_connector_errors

reset read connector errors

Returns: ComponentErrorsResetResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
bodyComponentErrorsResetRequest.OptionsReset options.True

[Back to top] [Back to API list] [Back to Models]

reset_transform_errors

reset transform errors

Returns: ComponentErrorsResetResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
bodyComponentErrorsResetRequest.OptionsReset options.True

[Back to top] [Back to API list] [Back to Models]

reset_write_connector_errors

reset write connector errors

Returns: ComponentErrorsResetResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
bodyComponentErrorsResetRequest.OptionsReset options.True

[Back to top] [Back to API list] [Back to Models]

share_connection

share connection to another data service

Returns: ShareConnectionResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringTrue
connection_idstringTrue
target_data_service_idstringTrue

[Back to top] [Back to API list] [Back to Models]

update_component_group

update component group in dataflow

Returns: ComponentGroupResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
component_group.idstringThe unique identifier of this component group.True
bodyComponentGroupComponent group details.True

[Back to top] [Back to API list] [Back to Models]

update_connection

update connection

Returns: UpdateConnectionResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringTrue
connection.id.valuestringTrue
bodyConnectionTrue

[Back to top] [Back to API list] [Back to Models]

update_container_image

update a container image definition

Returns: ContainerImageResponse

Parameters
NameTypeDescriptionRequired
labelstringa unique reference to the image.True
bodySpark.ContainerImageTrue

[Back to top] [Back to API list] [Back to Models]

update_data_feed

update data feed in dataflow

Returns: DataFeedResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
data_feed.idstringThe unique identifier of this data feed.True
bodyDataFeedData feed details.True

[Back to top] [Back to API list] [Back to Models]

update_data_feed_connector

update data feed connector in dataflow

Returns: DataFeedConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
data_feed_connector.idstringThe unique identifier of this data feed connector.True
bodyDataFeedConnectorData feed connector details.True

[Back to top] [Back to API list] [Back to Models]

update_dataflow

update dataflow

Returns: DataflowResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow.idstringThe unique identifier of this dataflow.True
bodyDataflowDataflow details.True

[Back to top] [Back to API list] [Back to Models]

update_data_service

update data service

Returns: DataServiceResponse

Parameters
NameTypeDescriptionRequired
idstringThe unique identifier of this data service.True
bodyDataServiceTrue

[Back to top] [Back to API list] [Back to Models]

update_data_service_credentials

update data service credentials

Returns: CredentialsResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
credential_idstringCredential id.True
bodyCredentialsComponent credentials.True

[Back to top] [Back to API list] [Back to Models]

update_data_service_runtime_environment

update runtime environment of a data service

Returns: DataServiceRuntimeEnvironmentResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringdata service id.True
bodyEnvironmentdefault runtime environment for the data service.True

[Back to top] [Back to API list] [Back to Models]

update_read_connector

update read connector in dataflow

Returns: ReadConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
read_connector.idstringThe unique identifier of this read connector.True
bodyReadConnectorRead connector to be created/updated.True

[Back to top] [Back to API list] [Back to Models]

update_read_connector_paused_state

pause or unpause a read connector

Returns: PausedStateResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
bodyUpdatePausedStateRequest.OptionsTrue
pausedbooleanSet to true to pause, or false(default) to unpause.False

[Back to top] [Back to API list] [Back to Models]

update_transform

update transform in dataflow

Returns: TransformResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
transform.idstringThe unique identifier of this transform.True
bodyTransformTransform details.True

[Back to top] [Back to API list] [Back to Models]

update_transform_paused_state

pause or unpause a transform

Returns: PausedStateResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
bodyUpdatePausedStateRequest.OptionsTrue
pausedbooleanSet to true to pause, or false(default) to unpause.False

[Back to top] [Back to API list] [Back to Models]

update_write_connector

update write connector in dataflow

Returns: WriteConnectorResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
write_connector.idstringThe unique identifier of this write connector.True
bodyWriteConnectorWrite connector details.True

[Back to top] [Back to API list] [Back to Models]

update_write_connector_paused_state

pause or unpause a write connector

Returns: PausedStateResponse

Parameters
NameTypeDescriptionRequired
data_service_idstringData service id.True
dataflow_idstringDataflow id.True
idstringComponent id.True
bodyUpdatePausedStateRequest.OptionsTrue
pausedbooleanSet to true to pause, or false(default) to unpause.False

[Back to top] [Back to API list] [Back to Models]

Models Documentation

ascend.protos.api.api_pb2.Component

A dataflow component.

NameTypeDescriptionRequired
idstringThe unique identifier of this component.False
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringComponent name.False
statusstringFalse
typestringLegacy type for component.False
updated_atstringLast updated timestamp.False
uuidstringComponent UUID.False
versionintegerFalse
inputComponentIDsstringInput scheduler-assigned component ids.False
inputsList<Component.Input>Input components, populated only for transforms.False
ComponentIDstringInternal ID assigned by scheduler.False
organizationDataServiceParent data service.False
prj_idstringDataflow id.False
projectDataflowParent dataflow.False
inputTypestringInput component legacy type - "source"/"view"/"sink".False
inputUUIDstringInput component UUID.False
openbooleanOpen for sharing with all data services - specific to data feeds.False
pubToRolesstringA list of data service roles to allow sharing - specific to data feeds.False
sourceSourceRead connector details, populated if component is a read connector.False
viewViewTransform details, populated if component is a transform.False
sinkSinkWrite connector details, populated if component is a write connector.False
contentList<Component.Input>Input component details, only populated for groups.False
schemaSchemaComponent schema.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.Component.Input

NameTypeDescriptionRequired
typestringLegacy type for input component.False
uuidstringInput component UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentErrorsResetRequest.Options

NameTypeDescriptionRequired
allbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentErrorsResetResponse

Component errors reset response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentErrorsResetResponse.ResponseFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentErrorsResetResponse.Response

NameTypeDescriptionRequired
uuidstringComponent UUID.False
statusstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentErrorsResponse

Component errors response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentErrorsResponse.ResponseList of partition errors.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentErrorsResponse.Response

NameTypeDescriptionRequired
errorsPartition.Error.SequenceFalse
uuidstringComponent UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentGroup

A component group.

NameTypeDescriptionRequired
idstringThe unique identifier of this component group.False
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringComponent group name.False
statusstringFalse
typestringLegacy type for group.False
updated_atstringLast updated timestamp.False
uuidstringGroup UUID.False
versionintegerFalse
organizationDataServiceParent data service.False
prj_idstringDataflow id.False
projectDataflowParent dataflow.False
contentList<ComponentGroup.Input>List of member components.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentGroup.Input

NameTypeDescriptionRequired
typestringLegacy type for constituent component.False
uuidstringConstituent component UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentGroupResponse

Component group response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentGroupComponent group details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentGroupsResponse

Component group listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<ComponentGroup>Component group details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentLineageResponse

Component lineage response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentLineageResponse.ResponseFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentLineageResponse.Node

NameTypeDescriptionRequired
data_service_idstringData service id.False
data_flow_idstringFalse
idstringComponent id.False
uuidstringComponent UUID.False
component_idstringFalse
component_typestringFalse
schemaListFalse
expressionstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentLineageResponse.Response

NameTypeDescriptionRequired
component_nodesList<ComponentLineageResponse.Node>False
component_edgesList<Lineage.Component.Edge>False
column_nodesList<Lineage.Column.Node>False
column_edgesList<Lineage.Column.Edge>False
uuidstringBackend UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentLogsResponse

NameTypeDescriptionRequired
statusstringFalse
dataComponentLogsResponse.ResponseFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentLogsResponse.Response

NameTypeDescriptionRequired
resultList<ComponentLogsResponse.Response.Stream>False
resultTypestringFalse
statsobjectFalse
next_page_cursorint64False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentLogsResponse.Response.Stream

NameTypeDescriptionRequired
streamobjectFalse
valuesList<List>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentPartitionsResponse

Component partitions response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentPartitionsResponse.ResponseList of Partition metadata results.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentPartitionsResponse.Response

NameTypeDescriptionRequired
dataPartition.Metadata.SequenceFalse
uuidstringComponent UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentRecordsResponse

Component records response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentRecordsResponse.ResponseRecords preview response.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentRecordsResponse.Response

NameTypeDescriptionRequired
dataResponseFalse
uuidstringComponent UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentSchemaResponse

Component schema response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentSchemaResponse.SchemaSchema response.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentSchemaResponse.Schema

NameTypeDescriptionRequired
schemastringComma-separated list of field names.False
uuidstringComponent UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentsResponse

Component listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<Component>List of components.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentStateResponse

Component state response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentStateResponse.StateComponent state details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentStateResponse.State

NameTypeDescriptionRequired
stateStateFalse
uuidstringComponent UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentStatisticsResponse

Component statistics response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentStatisticsResponse.StatisticsComponent statistics collection.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentStatisticsResponse.Statistics

NameTypeDescriptionRequired
statisticsStatistics.CollectionFalse
uuidstringComponent UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentWarningsResponse

Component warnings response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataComponentWarningsResponse.ResponseList of partition warnings.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ComponentWarningsResponse.Response

NameTypeDescriptionRequired
warningsPartition.Warning.SequenceFalse
uuidstringComponent UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ConnectionResponse

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerFalse
successbooleanFalse
dataConnectionFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ConnectionsResponse

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerFalse
successbooleanFalse
dataList<Connection>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ConnectionTypeResponse

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerFalse
successbooleanFalse
dataConnection.TypeFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ConnectionTypesResponse

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerFalse
successbooleanFalse
dataList<Connection.Type>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ContainerImageDeleteResponse

Container image deletion response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataContainerImageDeleteResponse.ContainerImageStatusContainerImage deletion status.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ContainerImageDeleteResponse.ContainerImageStatus

NameTypeDescriptionRequired
labelstringFalse
statusstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ContainerImageResponse

Container image creation response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataSpark.ContainerImageContainerImage definition.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ContainerImagesResponse

Container images response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<Spark.ContainerImage>list of ContainerImage definitions.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.CreateConnectionResponse

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerFalse
successbooleanFalse
dataConnectionFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.Credentials

component credentials

NameTypeDescriptionRequired
namestringCredential name that is unique across the Ascend environment.True
credential_idstringCredential id.False
credential_typestringFalse
canonical_credential_idstringFalse
is_privatebooleanFalse
owner_orgsList<Credentials.Org>False
credentialCredentialsFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.Credentials.Org

NameTypeDescriptionRequired
org_idstringFalse
org_namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.CredentialsResponse

Credentials creation/deletion response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataCredentialsFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataFeed

A data feed.

NameTypeDescriptionRequired
idstringThe unique identifier of this data feed.False
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringData feed name.False
statusstringFalse
typestringLegacy type for data feed.False
updated_atstringLast updated timestamp.False
uuidstringData feed UUID.False
versionintegerFalse
ComponentIDstringInternal ID assigned by scheduler.False
organizationDataServiceParent data service.False
prj_idstringDataflow id.False
projectDataflowParent dataflow.False
inputTypestringInput component legacy type - example - "view".False
inputUUIDstringInput component UUID.False
inputComponentIdstringInput scheduler-assigned component id.False
openbooleanOpen for sharing with all data services.False
pubToRolesstringA list of data service roles to allow sharing.False
schemaSchemaComponent schema.False
upstream_statusStatusFalse
org_idstringData service id.False
publicbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataFeedConnector

A data feed connector.

NameTypeDescriptionRequired
idstringThe unique identifier of this data feed connector.False
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringData feed connector name.False
statusstringFalse
typestringLegacy type for data feed connector.False
updated_atstringLast updated timestamp.False
uuidstringData feed connector UUID.False
versionintegerFalse
ComponentIDstringInternal ID assigned by scheduler.False
organizationDataServiceParent data service.False
prj_idstringDataflow id.False
projectDataflowParent dataflow.False
pubUUIDstringUUID of input data feed.False
schemaSchemaComponent schema.False
upstream_statusStatusFalse
stagesList<Stage>False
default_stageStageFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataFeedConnectorResponse

Data feed connector response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataDataFeedConnectorData feed connector details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataFeedConnectorsResponse

Data feed connector listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<DataFeedConnector>List of data feed connectors.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataFeedResponse

Data feed response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataDataFeedData feed details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataFeedsResponse

Data feed listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<DataFeed>List of data feeds.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.Dataflow

A dataflow.

NameTypeDescriptionRequired
idstringThe unique identifier of this dataflow.False
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringData flow name.False
org_idstringData service id.False
organizationDataServiceParent data service.False
statusstringFalse
typestringLegacy type.False
updated_atstringLast updated timestamp.False
uuidstringDataflow UUID.False
versionintegerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataflowErrorsResetRequest.Options

NameTypeDescriptionRequired
allbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataflowErrorsResetResponse

Data flow errors reset response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<DataflowErrorsResetResponse.Response>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataflowErrorsResetResponse.Response

NameTypeDescriptionRequired
uuidstringComponent UUID.False
errorstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataflowResponse

Dataflow response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataDataflowDataflow details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataflowsResponse

Dataflow listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<Dataflow>List of dataflows.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataService

A data service.

NameTypeDescriptionRequired
idstringThe unique identifier of this data service.True
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringData service name.False
statusstringFalse
typestringLegacy type.False
updated_atstringLast updated timestamp.False
uuidstringData service UUID.False
versionintegerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataServiceResponse

Data service response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataDataServiceData service.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataServiceRole

Data service role.

NameTypeDescriptionRequired
idstringThe unique identifier of this role.False
uuidstringRole UUID.False
namestringRole name.False
org_idstringData service id.False
org_namestringData service name.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataServiceRolesResponse

Data service roles response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<DataServiceRole>List of data service roles.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataServiceRuntimeEnvironmentResponse

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataEnvironmentdata service runtime environment.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DataServicesResponse

Data services listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<DataService>List of data services.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.Delete

Details in delete response.

NameTypeDescriptionRequired
uuidstringDeleted resource UUID.False
affectedint64False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DeleteCredentialsResponse

Data service credentials deletion response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataDeleteCredentialsResponse.CredentialsStatusFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DeleteCredentialsResponse.CredentialsStatus

NameTypeDescriptionRequired
credential_idstringFalse
statusstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.DeleteResponse

Data service deletion response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataDeletedeleted resource.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ListCredentialsResponse

Data service credentials listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<Credentials>List of credentials.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.PausedStateResponse

Read connector refresh response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataPausedStateResponse.ResponseFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.PausedStateResponse.Response

NameTypeDescriptionRequired
uuidstringFalse
statusstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ReadConnector

A read connector.

NameTypeDescriptionRequired
idstringThe unique identifier of this read connector.False
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringRead connector name.False
statusstringFalse
typestringLegacy type for component.False
updated_atstringLast updated timestamp.False
uuidstringRead connector UUID.False
versionintegerFalse
ComponentIDstringInternal ID assigned by scheduler.False
organizationDataServiceParent data service.False
prj_idstringDataflow id.False
projectDataflowParent dataflow.False
sourceSourceRead connector details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ReadConnectorRefreshResponse

Read connector refresh response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataReadConnectorRefreshResponse.ResponseFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ReadConnectorRefreshResponse.Response

NameTypeDescriptionRequired
uuidstringRead connector UUID.False
statusstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ReadConnectorResponse

Read connector response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataReadConnectorRead connector.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ReadConnectorsResponse

Read connector listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<ReadConnector>List of read connectors.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.ShareConnectionResponse

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerFalse
successbooleanFalse
datastringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.Status

NameTypeDescriptionRequired
imputed_priorityImputedPriorityImputed Processing Priority for component.False
stagesList<Stage>Stages for component.False
default_stageStageDefault stage of component.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.Transform

A transform.

NameTypeDescriptionRequired
idstringThe unique identifier of this transform.False
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringTransform name.False
statusstringFalse
typestringLegacy type for transform.False
updated_atstringLast updated timestamp.False
uuidstringTransform UUID.False
versionintegerFalse
ComponentIDstringInternal ID assigned by scheduler.False
organizationDataServiceParent data service.False
prj_idstringDataflow id.False
projectDataflowParent dataflow.False
inputComponentIDsstringInput scheduler-assigned component ids.False
inputsList<Transform.Input>List of input components.False
viewViewTransform details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.Transform.Input

NameTypeDescriptionRequired
typestringLegacy type for input component.False
uuidstringInput component UUID.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.TransformResponse

Transform response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataTransformTransform details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.TransformsResponse

Transform listing request.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<Transform>List of transforms.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.UpdateConnectionResponse

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerFalse
successbooleanFalse
dataConnectionFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.UpdatePausedStateRequest.Options

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.WriteConnector

A write connector.

NameTypeDescriptionRequired
idstringThe unique identifier of this write connector.False
created_atstringCreation timestamp.False
creator_UUIDstringFalse
deleted_atstringFalse
descriptionstringShort description.False
namestringWrite connector name.False
statusstringFalse
typestringLegacy type for write connector.False
updated_atstringLast updated timestamp.False
uuidstringWrite connector UUID.False
versionintegerFalse
ComponentIDstringInternal ID assigned by scheduler.False
organizationDataServiceParent data service.False
prj_idstringDataflow id.False
projectDataflowParent dataflow.False
inputComponentIDsstringInput scheduler-assigned component ids.False
inputTypestringInput component legacy type - "source"/"view".False
inputUUIDstringInput component UUID.False
sinkSinkWrite connector details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.WriteConnectorResponse

Write connector response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataWriteConnectorWrite connector details.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.api.api_pb2.WriteConnectorsResponse

Write connector listing response.

NameTypeDescriptionRequired
request_uristringFalse
response_codeintegerhttp response code.False
successbooleansuccess defined by a 2xx http response code.False
dataList<WriteConnector>List of write connectors.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.ascend.ascend_pb2.Struct

Direct copy of google/protobuf/struct.proto with added types. We
need to extend Value message to include other numeric types.

NameTypeDescriptionRequired
fieldsobjectUnordered map of dynamically typed values.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.ascend.ascend_pb2.Union

NameTypeDescriptionRequired
tagstringFalse
valueValueFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.ascend.ascend_pb2.Value

Value represents a dynamically typed value which can be either
null, a number, a string, a boolean, a recursive struct value, or a
list of values. A producer of value is expected to set one of that
variants, absence of any variant indicates an error.

The JSON representation for Value is JSON value.

NameTypeDescriptionRequired
null_valuestringRepresents a null value.False
number_valuenumberRepresents a double value.False
string_valuestringRepresents a string value.False
bool_valuebooleanRepresents a boolean value.False
struct_valueStructRepresents a structured value.False
list_valueValuesRepresents a repeated Value.False
byte_valueintegerFalse
short_valueintegerFalse
int_valueintegerFalse
long_valueint64False
timestamp_valuedate-timeRepresents a Timestamp value.False
date_valuedate-timeFalse
date_time_valuedate-timeFalse
decimal_valuestringFalse
binary_valuebyteFalse
union_valueUnionRepresents a tagged union value.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.ascend.ascend_pb2.Values

NameTypeDescriptionRequired
valuesList<Value>Repeated field of dynamically typed values.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.ApiDescriptor

NameTypeDescriptionRequired
idstringFalse
dataflow_idstringFalse
data_service_idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Descriptor

NameTypeDescriptionRequired
idIdFalse
typeTypeFalse
api_infoApiDescriptorFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Id

NameTypeDescriptionRequired
valuestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.ImputedPriority

NameTypeDescriptionRequired
weightnumberFalse
paused_byDescriptorFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Column.Edge

NameTypeDescriptionRequired
inputintegerFalse
output_component_expression_selectionLineage.SelectionFalse
expressionstringFalse
output_columnintegerFalse
output_componentintegerFalse
projectLineage.Column.Edge.ProjectFalse
aggregateLineage.Column.Edge.AggregateFalse
filterLineage.Column.Edge.FilterFalse
orderLineage.Column.Edge.OrderFalse
sinkLineage.Column.Edge.SinkFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Column.Edge.Aggregate

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Column.Edge.Filter

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Column.Edge.Order

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Column.Edge.Project

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Column.Edge.Sink

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Column.Node

NameTypeDescriptionRequired
namestringFalse
expressionstringFalse
component_node_indexintegerFalse
component_expression_selectionLineage.SelectionFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Component.Edge

NameTypeDescriptionRequired
inputintegerFalse
outputintegerFalse
output_expression_selectionLineage.SelectionFalse
expressionstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Selection

NameTypeDescriptionRequired
startLineage.Selection.PositionFalse
endLineage.Selection.PositionFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Lineage.Selection.Position

NameTypeDescriptionRequired
indexintegerFalse
lineintegerFalse
columnintegerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Priority

Processing Priority. Higher weight will be processed first.
If there is no weight, it will be processed as if its weight were 0.
Negative weights will be processed last.

NameTypeDescriptionRequired
weightnumberFalse
pausedbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Sink

Sink (Write Connector) component details.

NameTypeDescriptionRequired
inputIdId of the input component.False
containerContainerSink Container.False
assigned_priorityPriorityAssigned Processing Priority for component.False
statusSink.StatusSink Status. Read-only.False
bytesSink.ToBytesByte ouptut details. Required if sink has byte container.False
recordsSink.ToRecordsRecord output details. Required if sink has record container.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Sink.Status

Sink status.

NameTypeDescriptionRequired
imputed_priorityImputedPriorityImputed Processing Priority for component.False
stagesList<Stage>Stages for component.False
default_stageStageDefault stage of component.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Sink.ToBytes

Byte output details.

NameTypeDescriptionRequired
formatterOperatorOperator to transform record input data to byte output data.False
content_encodingContentEncodingContent encoding for output data.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Sink.ToRecords

Record output details.

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Source

Source (Read Connector) component details.

NameTypeDescriptionRequired
patternPatternSource list pattern.False
containerContainerSource container.False
updatePeriodicalPeriodicalSource Refresh Period.False
last_manual_refresh_timedate-timeLast Manual refresh for source ("Refresh Now").False
assigned_priorityPriorityProcessing Priority of Source.False
statusSource.StatusSource status. Read-only.False
aggregation_limitint64Dyanamic source aggregation limit. Dynamic Source aggregation will be disabled
if this is field is absent.False
bytesSource.FromBytesPopulated if input data is bytes.False
recordsSource.FromRecordsPopulated if input data is records.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Source.FromBytes

Byte data input details.

NameTypeDescriptionRequired
parserOperatorOperator to transform byte input data to records.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Source.FromRecords

Record data input details.

NameTypeDescriptionRequired
schemaMapSchema of record data.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Source.Status

Source status.

NameTypeDescriptionRequired
imputed_priorityImputedPriorityImputed Processing Priority for component.False
schemaSchemaComponent schema.False
stagesList<Stage>Stages for component.False
asr_statusSource.Status.AsrStatusDynamic source refresh status for source list.False
earliest_list_partition_refresh_timedate-timeOldest still-valid source refresh time.False
most_recent_list_partition_refresh_timedate-timeMost recent source refresh time.False
default_stageStageDefault stage of component.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Source.Status.AsrStatus

Dynamic source refresh status for source list.

NameTypeDescriptionRequired
refresh_suspended_untildate-timeTime for which re-listing is suspended due to unchanging results.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.State

The state of a component.

NameTypeDescriptionRequired
composite_stateStateRepresentative state, composited between the different stages of the component.False
stage_statesList<StageState>States for individual stages of the component.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Type

NameTypeDescriptionRequired
sourceType.SourceFalse
viewType.ViewFalse
sinkType.SinkFalse
pubType.PubFalse
subType.SubFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Type.Pub

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Type.Sink

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Type.Source

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Type.Sub

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.Type.View

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.View

View (Transform) component details.

NameTypeDescriptionRequired
inputList<Id>List of input component Ids. Required to be non-empty.False
operatorOperatorOperator to perform the transformation.False
assigned_priorityPriorityAssigned Processing Priority for component.False
statusView.StatusView Status. Read-only.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.component.component_pb2.View.Status

View Status.

NameTypeDescriptionRequired
imputed_priorityImputedPriorityImputed Processing Priority for component.False
schemaSchemaComponent schema.False
stagesList<Stage>Stages for component.False
default_stageStageDefault stage of component.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Credentials

NameTypeDescriptionRequired
type_idType.IdFalse
detailsobjectFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Id

NameTypeDescriptionRequired
valuestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field

NameTypeDescriptionRequired
namestringFalse
labelstringFalse
tooltip_textstringFalse
default_valueValueFalse
min_valueValueFalse
max_valueValueFalse
requiredbooleanFalse
avoid_checksumbooleanFalse
booleanType.Configuration.Field.BooleanFalse
dateType.Configuration.Field.DateFalse
doubleType.Configuration.Field.DoubleFalse
floatType.Configuration.Field.FloatFalse
intType.Configuration.Field.IntFalse
longType.Configuration.Field.LongFalse
stringType.Configuration.Field.StringFalse
structType.Configuration.Field.StructFalse
timestampType.Configuration.Field.TimestampFalse
unionType.Configuration.Field.UnionFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Boolean

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Date

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Double

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Float

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Int

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Long

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.String

NameTypeDescriptionRequired
multilinebooleanFalse
sensitivebooleanFalse
code_editorType.Configuration.Field.String.CodeEditorFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.String.CodeEditor

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Struct

NameTypeDescriptionRequired
fieldsList<Type.Configuration.Field>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Timestamp

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Field.Union

Union produces an protos.ascend.Union value, with tag equals to selected field name.
If a selected field is of Struct type, a UI should render a sub-form to collect details.
Otherwise, its default value is used as the specified value.

NameTypeDescriptionRequired
fieldsList<Type.Configuration.Field>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Configuration.Shape

NameTypeDescriptionRequired
fieldsList<Type.Configuration.Field>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Icon

NameTypeDescriptionRequired
urlstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Id

NameTypeDescriptionRequired
valuestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.connection.connection_pb2.Type.Status

NameTypeDescriptionRequired
connection_configuration_shapeType.Configuration.ShapeFalse
asset_configuration_shapeType.Configuration.ShapeFalse
credentials_configuration_shapeType.Configuration.ShapeFalse
iconType.IconFalse
require_credentialsbooleanFalse
category_idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.content_encoding.content_encoding_pb2.Compress

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.content_encoding.content_encoding_pb2.ContentEncoding

NameTypeDescriptionRequired
identityIdentityFalse
compressCompressFalse
deflateDeflateFalse
gzipGzipFalse
lzoLzoFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.content_encoding.content_encoding_pb2.Deflate

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.content_encoding.content_encoding_pb2.Gzip

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.content_encoding.content_encoding_pb2.Identity

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.content_encoding.content_encoding_pb2.Lzo

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.core.core_pb2.ListValue

NameTypeDescriptionRequired
valuesList<Value>Repeated field of dynamically typed values.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.core.core_pb2.LogRequest

NameTypeDescriptionRequired
labelstringlabel identifying logs.False
range_startint64start of log query: UTC unix timestamp in nanoseconds.False
range_endint64end of log query: UTC unix timestamp in nanoseconds.False
jwtstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.core.core_pb2.Periodical

NameTypeDescriptionRequired
periodstringFalse
offsetstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.core.core_pb2.Statistics

In the scheduler DB. The extra oneofs are so we can differentiate
default values from real values.

NameTypeDescriptionRequired
cardinalityuint64False
sumValueFalse
distinct_cardinalityuint64False
approximate_distinct_cardinalityuint64False
distinct_valuesListValueFalse
distinct_values_available_for_all_non_empty_leavesbooleanFalse
schemaSchemaFalse
minimumValueFalse
maximumValueFalse
null_countuint64False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.core.core_pb2.Statistics.Collection

NameTypeDescriptionRequired
dimensionsobjectFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.core.core_pb2.Struct

Direct copy of google/protobuf/struct.proto with added types. We
need to extend Value message to include other numeric types.

NameTypeDescriptionRequired
fieldsobjectUnordered map of dynamically typed values.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.core.core_pb2.Union

NameTypeDescriptionRequired
tagstringFalse
valueValueFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.core.core_pb2.Value

Value represents a dynamically typed value which can be either
null, a number, a string, a boolean, a recursive struct value, or a
list of values. A producer of value is expected to set one of that
variants, absence of any variant indicates an error.

The JSON representation for Value is JSON value.

NameTypeDescriptionRequired
null_valuestringRepresents a null value.False
number_valuenumberRepresents a double value.False
string_valuestringRepresents a string value.False
bool_valuebooleanRepresents a boolean value.False
struct_valueStructRepresents a structured value.False
list_valueListValueRepresents a repeated Value.False
byte_valueintegerFalse
short_valueintegerFalse
int_valueintegerFalse
long_valueint64False
timestamp_valuedate-timeRepresents a Timestamp value.False
date_valuedate-timeFalse
date_time_valuedate-timeFalse
decimal_valuestringFalse
binary_valuebyteFalse
union_valueUnionRepresents a tagged union value.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.environment.environment_pb2.Spark.ContainerImage

Runtime container image.

NameTypeDescriptionRequired
labelstringa unique reference to the image.False
namestringfully qualified image name including registry name, repository, and tag.False
runtimestringspark runtime - has the format "scala:<scala_ver>;spark:<spark_ver>", example - scala:2.12;spark:3.1.0.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Partition.Error

NameTypeDescriptionRequired
faultFaultFalse
recorded_atdate-timeLast recorded error timestamp.False
failedAttemptsint64Total number of times this partition has failed.False
log_requestLogRequestParameters to request logs.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Partition.Error.Sequence

NameTypeDescriptionRequired
errorList<Partition.Error>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Partition.Metadata

NameTypeDescriptionRequired
statePartition.StateState for partition.False
statisticsStatistics.CollectionStatistics for partition.False
errorsList<Partition.Error>Errors for partition.False
request_tokenstringUnique token to specify partition in requests.False
failure_statisticsStatistics.CollectionStatistics for failures associated with partition.False
labelstringUnique label to identify partition.False
log_requestLogRequestParameters to request logs for partition.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Partition.Metadata.Sequence

NameTypeDescriptionRequired
metadataList<Partition.Metadata>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Partition.State

State for Partition.

NameTypeDescriptionRequired
totalint64Total number of partitions, empty if unknown.False
up_to_dateint64Number of up to date partitions.False
runningint64Number of running partitions.False
retryingint64Number of retrying partitions.False
errorsList<Partition.Error>False
warningsList<Fault>False
num_warningsint64Number of warnings.False
eligible_for_retryint64Number of partitions which have failed and can be retried now.False
ineligible_for_retryint64Number of partitions which have failed and can be retried later.False
never_eligible_for_retryint64Number of partitions which have failed and exhausted all retries.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Partition.Warning

NameTypeDescriptionRequired
faultFaultFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Partition.Warning.Sequence

NameTypeDescriptionRequired
warningList<Partition.Warning>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage

NameTypeDescriptionRequired
analyzeStage.AnalyzeFalse
latestStage.LatestFalse
listStage.ListFalse
loadStage.LoadFalse
parseStage.ParseFalse
queryStage.QueryFalse
readStage.ReadFalse
reshapeStage.ReshapeFalse
staging_listStage.StagingListFalse
staging_sweepStage.StagingSweepFalse
staging_writeStage.StagingWriteFalse
sweepStage.SweepFalse
transformStage.TransformFalse
writeStage.WriteFalse
write_manifestStage.WriteManifestFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Analyze

NameTypeDescriptionRequired
indexintegerFalse
dimensionstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Latest

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.List

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Load

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Parse

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Query

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Read

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Reshape

NameTypeDescriptionRequired
indexintegerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.StagingList

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.StagingSweep

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.StagingWrite

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Sweep

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Transform

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.Write

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.Stage.WriteManifest

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.StageState

Message associating a specific stage with its state.

NameTypeDescriptionRequired
stageStageFalse
stateStateFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State

Expression state.

NameTypeDescriptionRequired
partition_statePartition.StateAggregated partition states of expression.False
textstringState label.False
errorsList<Fault>Errors for state. Only populated in webhooks, when in error state.False
maybe_committed_partition_countint64If results are available, the number of comitted partitions.False
up_to_dateState.UpToDatePopulated if state is Up To Date. This means that the component has completed processing
with respect to all updates to itself, and its inputs.False
runningState.RunningPopulated if state is Running. This means that there are currently tasks running
for the component that will contribute to making the component Up To Date.False
out_of_dateState.OutOfDatePopulated if state is out of date. This means that the component is not Up To Date
or Running, wrapping more specific details inside the OutOfDate message.False
errorState.ErrorPopulated if the state is in Error. This means that the component has errors which have
exhausted all retries.False
pendingState.PendingFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.Error

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.OutOfDate

NameTypeDescriptionRequired
blockedState.OutOfDate.BlockedPopulated if expression is blocked. This means there is an upstream
expression with an error preventing processing of this expression.False
pendingState.OutOfDate.PendingPopulated if expression is pending on input This means that there is an
upstream expression processing data updates, but there exist results.False
readyState.OutOfDate.ReadyPopulated if expression is ready to Run. This means that all of its inputs
are up to date, however no tasks are currently running.False
waitingState.OutOfDate.WaitingPopulated if expression is Waiting. This means that it is waiting for upstream.
processing to finish, and no previous results are available.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.OutOfDate.Blocked

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.OutOfDate.Pending

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.OutOfDate.Ready

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.OutOfDate.Waiting

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.Pending

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.Running

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.expression.expression_pb2.State.UpToDate

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Aws.Credentials

NameTypeDescriptionRequired
idstringFalse
secretstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Aws.Redshift.Credentials

NameTypeDescriptionRequired
usernamestringFalse
passwordstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Aws.Redshift.Database

NameTypeDescriptionRequired
credentialsAws.Redshift.CredentialsFalse
jdbc_urlstringFalse
default_to_sslbooleanFalse
credential_idCredentials.IdFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Aws.Redshift.Table

NameTypeDescriptionRequired
databaseAws.Redshift.DatabaseFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Aws.S3.Bucket

NameTypeDescriptionRequired
credentialsAws.CredentialsFalse
namestringFalse
regionstringFalse
credential_idCredentials.IdFalse
sse_kms_arnstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Aws.S3.Object

NameTypeDescriptionRequired
bucketAws.S3.BucketFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Azure.Abs.Blob

NameTypeDescriptionRequired
containerAzure.Abs.ContainerFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Azure.Abs.Container

NameTypeDescriptionRequired
accountAzure.AccountFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Azure.Account

NameTypeDescriptionRequired
credentialsAzure.CredentialsFalse
namestringFalse
credential_idCredentials.IdFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Azure.Credentials

NameTypeDescriptionRequired
shared_keyAzure.Credentials.SharedKeyFalse
service_principalAzure.Credentials.ServicePrincipalFalse
service_principal_jsonAzure.Credentials.ServicePrincipalJsonFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Azure.Credentials.ServicePrincipal

NameTypeDescriptionRequired
client_idstringFalse
client_secretstringFalse
tenantstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Azure.Credentials.ServicePrincipalJson

NameTypeDescriptionRequired
json_keystringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Azure.Credentials.SharedKey

NameTypeDescriptionRequired
keystringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Code

NameTypeDescriptionRequired
languageCode.LanguageFalse
sourceCode.SourceFalse
environmentEnvironmentFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Code.Source

NameTypeDescriptionRequired
inlinestringFalse
objectCode.Source.ObjectFalse
binarybyteFalse
built_inFalse
override_fingerprintstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Code.Source.Object

NameTypeDescriptionRequired
objectObjectFalse
version_shabyteFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Connection.Asset

NameTypeDescriptionRequired
connectionIdFalse
detailsobjectFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Credentials.Id

NameTypeDescriptionRequired
valuestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Executable

NameTypeDescriptionRequired
codeCodeFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.File.Object

NameTypeDescriptionRequired
pathFile.PathFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.File.Path

NameTypeDescriptionRequired
prefixstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Function.Credentials

NameTypeDescriptionRequired
secretstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Function.CredentialsConfiguration

NameTypeDescriptionRequired
idCredentials.IdFalse
credentialsFunction.CredentialsFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Function.Object

NameTypeDescriptionRequired
storeFunction.StoreFalse
metadataStructFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Function.Store

NameTypeDescriptionRequired
executableExecutableFalse
credentials_configurationFunction.CredentialsConfigurationFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Gcp.BigQuery.Dataset

NameTypeDescriptionRequired
projectGcp.ProjectFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Gcp.BigQuery.Table

NameTypeDescriptionRequired
datasetGcp.BigQuery.DatasetFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Gcp.Credentials

NameTypeDescriptionRequired
json_keystringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Gcp.Gcs.Bucket

NameTypeDescriptionRequired
credentialsGcp.CredentialsFalse
namestringFalse
credential_idCredentials.IdFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Gcp.Gcs.Object

NameTypeDescriptionRequired
bucketGcp.Gcs.BucketFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Gcp.Project

NameTypeDescriptionRequired
credentialsGcp.CredentialsFalse
namestringFalse
credential_idCredentials.IdFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.MsSqlServer.Credentials

NameTypeDescriptionRequired
sql_authenticationMsSqlServer.Credentials.SqlAuthenticationFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.MsSqlServer.Credentials.SqlAuthentication

NameTypeDescriptionRequired
userstringFalse
passwordstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.MsSqlServer.Database

NameTypeDescriptionRequired
serverMsSqlServer.ServerFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.MsSqlServer.Server

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
credentialsMsSqlServer.CredentialsFalse
namestringFalse
portintegerFalse
use_encryptionbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.MsSqlServer.Table

NameTypeDescriptionRequired
databaseMsSqlServer.DatabaseFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Mysql.Credentials

NameTypeDescriptionRequired
usernamestringFalse
passwordstringFalse
certificateMysql.TlsCertificateFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Mysql.Database

NameTypeDescriptionRequired
serverMysql.ServerFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Mysql.Partition

NameTypeDescriptionRequired
tableMysql.TableFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Mysql.Server

NameTypeDescriptionRequired
credentialsMysql.CredentialsFalse
hoststringFalse
portintegerFalse
credential_idCredentials.IdFalse
disable_tlsbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Mysql.Table

NameTypeDescriptionRequired
databaseMysql.DatabaseFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Mysql.TlsCertificate

NameTypeDescriptionRequired
server_castringFalse
client_certstringFalse
client_keystringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Object

NameTypeDescriptionRequired
redshiftAws.Redshift.TableFalse
s3Aws.S3.ObjectFalse
absAzure.Abs.BlobFalse
fileFile.ObjectFalse
functionFunction.ObjectFalse
big_queryGcp.BigQuery.TableFalse
gcsGcp.Gcs.ObjectFalse
ms_sql_serverMsSqlServer.TableFalse
mysql_partitionMysql.PartitionFalse
mysqlMysql.TableFalse
snowflakeSnowflake.TableFalse
sparkSpark.TableFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Snowflake.Credentials

NameTypeDescriptionRequired
usernamestringFalse
passwordstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Snowflake.Database

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
credentialsSnowflake.CredentialsFalse
accountstringFalse
warehousestringFalse
databasestringFalse
snowflake_schemastringFalse
rolestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Snowflake.Table

NameTypeDescriptionRequired
databaseSnowflake.DatabaseFalse
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Spark.Store

NameTypeDescriptionRequired
s3Aws.S3.BucketFalse
gcsGcp.Gcs.BucketFalse
absAzure.Abs.ContainerFalse
fileFile.PathFalse
record_connectionConnection.AssetFalse
object_suffixstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.external.external_pb2.Spark.Table

NameTypeDescriptionRequired
storeSpark.StoreFalse
object_prefixstringFalse
metadataStructFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.AwaitingService

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.AwsBadCredentials

NameTypeDescriptionRequired
awsAws.S3.LocationFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.AwsError

NameTypeDescriptionRequired
awsAws.S3.LocationFalse
user_friendly_messagestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.Backtrace

NameTypeDescriptionRequired
stack_frameListFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.BadLocationTemplate

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.BadSql

NameTypeDescriptionRequired
sqlstringFalse
field_not_foundBadSql.FieldNotFoundFalse
parse_errorBadSql.ParseErrorFalse
bq_invalid_queryBqInvalidQueryFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.BadSql.FieldNotFound

NameTypeDescriptionRequired
fieldstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.BadSql.ParseError

NameTypeDescriptionRequired
markBadSql.ParseError.MarkFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.BadSql.ParseError.Mark

NameTypeDescriptionRequired
linenoint64False
colnoint64False
linestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.BqAccessError

NameTypeDescriptionRequired
messagestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.BqInvalidQuery

NameTypeDescriptionRequired
projectstringTODO Move under Badql, will drop after all code deployed.False
datasetstringFalse
sqlstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.BqLoadAvroEof

NameTypeDescriptionRequired
messagestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.CommandRejected

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.ConnectionFailed

NameTypeDescriptionRequired
error_messagestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.CredentialsMismatch

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.DataLimitExceeded

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.ExternalServiceUnavailable

NameTypeDescriptionRequired
s3Aws.S3False
bigqueryGcp.BigQueryFalse
gcsGcp.GcsFalse
mysqlMysqlFalse
redshiftAws.RedshiftFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.Fault

NameTypeDescriptionRequired
descriptionstringFalse
backtraceBacktraceFalse
causeFaultFalse
timestampdate-timeFalse
awaiting_serviceAwaitingServiceFalse
aws_bad_credentialsAwsBadCredentialsFalse
aws_errorAwsErrorFalse
bad_sqlBadSqlFalse
bqAccessErrorBqAccessErrorFalse
bq_invalid_queryBqInvalidQueryFalse
bqLoadAvroEofBqLoadAvroEofFalse
command_rejectedCommandRejectedFalse
connection_failedConnectionFailedFalse
credentials_mismatchCredentialsMismatchFalse
data_limit_exceededDataLimitExceededFalse
external_service_unavailableExternalServiceUnavailableFalse
feature_unavailableFeatureUnavailableFalse
file_not_foundFileNotFoundFalse
fragment_not_foundFragmentNotFoundFalse
function_errorFunctionErrorFalse
grpc_message_too_largeGrpcMessageTooLargeFalse
illegal_argumentIllegalArgumentFalse
image_pull_failedImagePullFailedFalse
insufficient_resourcesInsufficientResourcesFalse
internal_errorInternalErrorFalse
internal_service_unavailableInternalServiceUnavailableFalse
invalid_configInvalidConfigFalse
invalid_credentials_on_identity_assignmentInvalidCredentialsOnIdentityAssignmentFalse
invalid_schemaInvalidSchemaFalse
mysql_bad_credentialsMysqlBadCredentialsFalse
mysql_db_does_not_existsMysqlDbDoesNotExistFalse
mysql_errorMysqlErrorFalse
no_such_componentNoSuchComponentFalse
no_such_taskNoSuchTaskFalse
object_not_foundObjectNotFoundFalse
task_canceledTaskCanceledFalse
task_not_foundTaskNotFoundFalse
task_rejectedTaskRejectedFalse
too_many_bad_recordsTooManyBadRecordsFalse
udp_errorUdpErrorFalse
upstream_errorUpstreamErrorFalse
bad_location_templateBadLocationTemplateFalse
list_content_limit_exceededListContentLimitExceededFalse
extendedprotobuf.AnyFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.FeatureUnavailable

NameTypeDescriptionRequired
namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.FileNotFound

NameTypeDescriptionRequired
filenamestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.FragmentNotFound

NameTypeDescriptionRequired
fshastringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.FunctionError

NameTypeDescriptionRequired
retryablebooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.GrpcMessageTooLarge

NameTypeDescriptionRequired
sizeint64False
limitint64False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.IllegalArgument

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.ImagePullFailed

NameTypeDescriptionRequired
image_namestringFalse
messagestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.InsufficientResources

NameTypeDescriptionRequired
memoryInsufficientResources.MemoryFalse
cpuInsufficientResources.CpuFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.InsufficientResources.Cpu

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.InsufficientResources.Memory

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.InternalError

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.InternalServiceUnavailable

NameTypeDescriptionRequired
retry_afterstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.InvalidConfig

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.InvalidCredentialsOnIdentityAssignment

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.InvalidSchema

NameTypeDescriptionRequired
schemaSchemaFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.ListContentLimitExceeded

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.MysqlBadCredentials

NameTypeDescriptionRequired
mysqlMysql.LocationFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.MysqlDbDoesNotExist

NameTypeDescriptionRequired
mysqlMysql.LocationFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.MysqlError

NameTypeDescriptionRequired
mysqlMysql.LocationFalse
user_friendly_messagestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.NoSuchComponent

NameTypeDescriptionRequired
idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.NoSuchTask

NameTypeDescriptionRequired
idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.ObjectNotFound

NameTypeDescriptionRequired
missing_objectObjectFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.TaskCanceled

NameTypeDescriptionRequired
task_idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.TaskNotFound

NameTypeDescriptionRequired
task_idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.TaskRejected

NameTypeDescriptionRequired
task_idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.TooManyBadRecords

NameTypeDescriptionRequired
countint64False
errorList<TooManyBadRecords.ParseError>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.TooManyBadRecords.ParseError

NameTypeDescriptionRequired
lineint64False
columnint64False
messagestringFalse
file_namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.UdpError

NameTypeDescriptionRequired
user_friendly_messagestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.fault.fault_pb2.UpstreamError

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Avro

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Avro.Formatter

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Avro.Parser

NameTypeDescriptionRequired
schemaMapFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Columns

NameTypeDescriptionRequired
parsing_specificationsList<Columns.ParsingSpecification>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Columns.ParsingSpecification

NameTypeDescriptionRequired
source_namestringFalse
target_namestringFalse
null_is_invalidbooleanFalse
invalid_value_handlingColumns.ParsingSpecification.InvalidValueHandlingFalse
booleanBoolean.ParserFalse
dateDate.ParserFalse
doubleDouble.ParserFalse
floatFloat.ParserFalse
intInt.ParserFalse
longLong.ParserFalse
stringString.ParserFalse
timestampTimestamp.ParserFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Columns.ParsingSpecification.InvalidValueHandling

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Columns.ParsingSpecification.InvalidValueHandling.Fail

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Columns.ParsingSpecification.InvalidValueHandling.SkipRecord

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Columns.ParsingSpecification.InvalidValueHandling.UseDefault

NameTypeDescriptionRequired
default_valuestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Format

NameTypeDescriptionRequired
avroAvroFalse
grokGrokFalse
jsonJsonFalse
parquetParquetFalse
xsvXsvFalse
orcOrcFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Grok

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Grok.Parser

NameTypeDescriptionRequired
patternstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Json

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Json.Formatter

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Json.Parser

NameTypeDescriptionRequired
columnsColumnsFalse
flatten_nestedbooleanFalse
max_warn_countint64False
namespace_delimiterstringFalse
schemaMapFalse
max_error_countintegerFalse
max_error_percentage_pointsnumberFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Formatter

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Parser

NameTypeDescriptionRequired
docker_imageLambda.Parser.DockerImageFalse
codeLambda.Parser.CodeFalse
metadatastringFalse
schemaMapFalse
max_error_countintegerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Parser.Code

NameTypeDescriptionRequired
languageLambda.Parser.LanguageFalse
inlinestringFalse
fileLambda.Parser.FileFalse
override_fingerprintstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Parser.DockerImage

NameTypeDescriptionRequired
image_namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Parser.File

NameTypeDescriptionRequired
uristringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Parser.Language

NameTypeDescriptionRequired
pythonLambda.Parser.Language.PythonFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Parser.Language.Python

NameTypeDescriptionRequired
pip_packageListFalse
v2Lambda.Parser.Language.Python.V2False
v3Lambda.Parser.Language.Python.V3False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Parser.Language.Python.V2

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Lambda.Parser.Language.Python.V3

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Orc

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Orc.Formatter

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Parquet

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Parquet.Formatter

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Parquet.Parser

NameTypeDescriptionRequired
schemaMapFalse
namespace_delimiterstringFalse
require_no_missing_fields_in_parquetbooleanFalse
require_no_extra_fields_in_parquetbooleanFalse
require_field_type_match_schemabooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Xsv

NameTypeDescriptionRequired
optionsXsv.OptionsFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Xsv.Formatter

NameTypeDescriptionRequired
delimiterstringFalse
include_headerbooleanFalse
line_endingLineEndingFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Xsv.Options

NameTypeDescriptionRequired
delimiterstringFalse
include_headerbooleanFalse
line_endingLineEndingFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.format.format_pb2.Xsv.Parser

NameTypeDescriptionRequired
columnsColumnsFalse
comment_prefixstringFalse
delimiterstringFalse
escape_characterstringFalse
header_line_countint64False
max_warn_countint64False
quote_characterstringFalse
headers_to_columnsobjectFalse
max_error_countint64False
max_error_percentage_pointsnumberFalse
schemaMapFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.function.function_pb2.Code.Language

NameTypeDescriptionRequired
pythonCode.Language.PythonFalse
javaCode.Language.JavaFalse
sqlCode.Language.SqlFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.function.function_pb2.Code.Language.Java

NameTypeDescriptionRequired
class_namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.function.function_pb2.Code.Language.Python

NameTypeDescriptionRequired
pip_packageListFalse
class_namestringFalse
v2Code.Language.Python.Version.V2False
v3Code.Language.Python.Version.V3False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.function.function_pb2.Code.Language.Python.Version.V2

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.function.function_pb2.Code.Language.Python.Version.V3

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.function.function_pb2.Code.Language.Sql

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.function.function_pb2.Environment

Runtime environment for user code.

NameTypeDescriptionRequired
spark_runtimestringspark runtime - has the format "scala:<scala_ver>;spark:<spark_ver>", example - scala:2.12;spark:3.1.0.False
container_image_labelstringa reference to an image that is registered in the Ascend environment.False
container_imageSpark.ContainerImagecontainer image.False
parent_spark_environmentEnvironment.Defaultdefault spark environment inherited from parent entity - for example, the data service or site-level default.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.function.function_pb2.Environment.Default

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.google.google_pb2.api.HttpBody

Message that represents an arbitrary HTTP body. It should only be used for
payload formats that can't be represented as JSON, such as raw binary or
an HTML page.

This message can be used both in streaming and non-streaming API methods in
the request as well as the response.

It can be used as a top-level request field, which is convenient if one
wants to extract parameters from either the URL or HTTP template into the
request fields and also want access to the raw HTTP body.

Example:

message GetResourceRequest {
  // A unique request id.
  string request_id = 1;

  // The raw HTTP body is bound to this field.
  google.api.HttpBody http_body = 2;
}

service ResourceService {
  rpc GetResource(GetResourceRequest) returns (google.api.HttpBody);
  rpc UpdateResource(google.api.HttpBody) returns
  (google.protobuf.Empty);
}

Example with streaming methods:

service CaldavService {
  rpc GetCalendar(stream google.api.HttpBody)
    returns (stream google.api.HttpBody);
  rpc UpdateCalendar(stream google.api.HttpBody)
    returns (stream google.api.HttpBody);
}

Use of this type only changes how the request and response bodies are
handled, all other features will continue to work unchanged.

NameTypeDescriptionRequired
content_typestringThe HTTP Content-Type header value specifying the content type of the body.False
databyteThe HTTP request/response body as raw binary.False
extensionsList<protobuf.Any>Application specific response metadata. Must be set in the first response
for streaming APIs.False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.google.google_pb2.protobuf.Any

Any contains an arbitrary serialized protocol buffer message along with a
URL that describes the type of the serialized message.

Protobuf library provides support to pack/unpack Any values in the form
of utility functions or additional generated methods of the Any type.

Example 1: Pack and unpack a message in C++.

Foo foo = ...;
Any any;
any.PackFrom(foo);
...
if (any.UnpackTo(&foo)) {
  ...
}

Example 2: Pack and unpack a message in Java.

Foo foo = ...;
Any any = Any.pack(foo);
...
if (any.is(Foo.class)) {
  foo = any.unpack(Foo.class);
}

Example 3: Pack and unpack a message in Python.

foo = Foo(...)
any = Any()
any.Pack(foo)
...
if any.Is(Foo.DESCRIPTOR):
  any.Unpack(foo)
  ...

Example 4: Pack and unpack a message in Go

 foo := &pb.Foo{...}
 any, err := anypb.New(foo)
 if err != nil {
   ...
 }
 ...
 foo := &pb.Foo{}
 if err := any.UnmarshalTo(foo); err != nil {
   ...
 }

The pack methods provided by protobuf library will by default use
'type.googleapis.com/full.type.name' as the type URL and the unpack
methods only use the fully qualified type name after the last '/'
in the type URL, for example "foo.bar.com/x/y.z" will yield type
name "y.z".

JSON

The JSON representation of an Any value uses the regular
representation of the deserialized, embedded message, with an
additional field @type which contains the type URL. Example:

package google.profile;
message Person {
  string first_name = 1;
  string last_name = 2;
}

{
  "@type": "type.googleapis.com/google.profile.Person",
  "firstName": <string>,
  "lastName": <string>
}

If the embedded message type is well-known and has a custom JSON
representation, that representation will be embedded adding a field
value which holds the custom JSON in addition to the @type
field. Example (for message [google.protobuf.Duration][]):

{
  "@type": "type.googleapis.com/google.protobuf.Duration",
  "value": "1.212s"
}
NameTypeDescriptionRequired
type_urlstringA URL/resource name that uniquely identifies the type of the serialized

protocol buffer message. This string must contain at least
one "/" character. The last segment of the URL's path must represent
the fully qualified name of the type (as in
path/google.protobuf.Duration). The name should be in a canonical form
(e.g., leading "." is not accepted).

In practice, teams usually precompile into the binary all types that they
expect it to use in the context of Any. However, for URLs which use the
scheme http, https, or no scheme, one can optionally set up a type
server that maps type URLs to message definitions as follows:

  • If no scheme is provided, https is assumed.
  • An HTTP GET on the URL must yield a [google.protobuf.Type][]
    value in binary format, or produce an error.
  • Applications are allowed to cache lookup results based on the
    URL, or have them precompiled into a binary to avoid any
    lookup. Therefore, binary compatibility needs to be preserved
    on changes to types. (Use versioned type names to manage
    breaking changes.)

Note: this functionality is not currently available in the official
protobuf release, and it is not used for type URLs beginning with
type.googleapis.com.

Schemes other than http, https (or the empty scheme) might be
used with implementation specific semantics. | False
value | byte | Must be a valid serialized protocol buffer of the above specified type. | False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.google.google_pb2.protobuf.NullValue

NullValue is a singleton enumeration to represent the null value for the
Value type union.

The JSON representation for NullValue is JSON null.

  • NULL_VALUE: Null value.

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.grpc.grpc_pb2.gateway.runtime.Error

NameTypeDescriptionRequired
errorstringFalse
codeintegerFalse
messagestringFalse
detailsList<protobuf.Any>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.grpc.grpc_pb2.gateway.runtime.StreamError

NameTypeDescriptionRequired
grpc_codeintegerFalse
http_codeintegerFalse
messagestringFalse
http_statusstringFalse
detailsList<protobuf.Any>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Aws.Credentials

NameTypeDescriptionRequired
idstringFalse
secretstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Aws.Redshift

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Aws.Redshift.Container

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
jdbc_urlstringFalse
staging_containerAws.S3.ContainerFalse
location_templatestringFalse
default_to_sslbooleanFalse
update_policyUpdatePolicyFalse
iam_role_arnstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Aws.Redshift.Credentials

NameTypeDescriptionRequired
usernamestringFalse
passwordstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Aws.S3

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Aws.S3.Container

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
regionstringFalse
bucketstringFalse
prefixstringFalse
delimiterstringFalse
disable_random_location_infixbooleanFalse
location_templatestringFalse
location_suffixstringFalse
manifestManifestFalse
write_part_filesbooleanFalse
use_flat_part_file_layoutbooleanFalse
sse_kms_arnstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Aws.S3.Location

NameTypeDescriptionRequired
containerAws.S3.ContainerFalse
object_namestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Azure.Abs.Container

NameTypeDescriptionRequired
account_namestringFalse
credential_idCredentials.IdFalse
containerstringFalse
prefixstringFalse
delimiterstringFalse
disable_random_location_infixbooleanFalse
location_templatestringFalse
location_suffixstringFalse
manifestManifestFalse
write_part_filesbooleanFalse
use_flat_part_file_layoutbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Azure.Credentials

NameTypeDescriptionRequired
shared_keyAzure.Credentials.SharedKeyFalse
service_principalAzure.Credentials.ServicePrincipalFalse
service_principal_jsonAzure.Credentials.ServicePrincipalJsonFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Azure.Credentials.ServicePrincipal

NameTypeDescriptionRequired
client_idstringFalse
client_secretstringFalse
tenantstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Azure.Credentials.ServicePrincipalJson

NameTypeDescriptionRequired
json_keystringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Azure.Credentials.SharedKey

NameTypeDescriptionRequired
keystringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Code

NameTypeDescriptionRequired
languageCode.LanguageFalse
sourceCode.SourceFalse
environmentEnvironmentFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Code.Source

NameTypeDescriptionRequired
inlinestringFalse
locationCode.Source.LocationFalse
built_inFalse
override_fingerprintstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Code.Source.Location

NameTypeDescriptionRequired
containerContainerFalse
patternPatternFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Connection

NameTypeDescriptionRequired
type_idType.IdFalse
idIdFalse
namestringFalse
detailsobjectFalse
credential_idCredentials.IdFalse
entity_metadataConnection.EntityMetadataFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Connection.Asset.Record

NameTypeDescriptionRequired
connection_idIdFalse
detailsobjectFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Connection.EntityMetadata

NameTypeDescriptionRequired
created_atstringFalse
updated_atstringFalse
owner_orgsList<Connection.EntityMetadata.Org>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Connection.EntityMetadata.Org

NameTypeDescriptionRequired
org_idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Connection.Type

NameTypeDescriptionRequired
idType.IdFalse
namestringFalse
versionstringFalse
executableExecutableFalse
alternative_credential_typeList<Type.Id>False
statusType.StatusFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Container

NameTypeDescriptionRequired
redshiftAws.Redshift.ContainerFalse
s3Aws.S3.ContainerFalse
absAzure.Abs.ContainerFalse
record_connectionConnection.Asset.RecordFalse
fileFile.ContainerFalse
byte_functionFunction.Container.ByteFalse
record_functionFunction.Container.RecordFalse
big_queryGcp.BigQuery.ContainerFalse
gcsGcp.Gcs.ContainerFalse
immediateImmediate.ContainerFalse
ms_sql_serverMsSqlServer.ContainerFalse
mysqlMysql.ContainerFalse
mysql_partitionMysql.Partition.ContainerFalse
snowflakeSnowflake.ContainerFalse
sparkSpark.ContainerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Container.ShaOverrides

NameTypeDescriptionRequired
cshabyteFalse
read_io_shabyteFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Credentials

NameTypeDescriptionRequired
idCredentials.IdFalse
awsAws.CredentialsFalse
redshiftAws.Redshift.CredentialsFalse
azureAzure.CredentialsFalse
functionFunction.CredentialsFalse
gcpGcp.CredentialsFalse
ms_sql_serverMsSqlServer.CredentialsFalse
mysqlMysql.CredentialsFalse
snowflakeSnowflake.CredentialsFalse
connectionCredentialsFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Credentials.Id

NameTypeDescriptionRequired
valuestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Executable

NameTypeDescriptionRequired
codeCodeFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.File.Container

NameTypeDescriptionRequired
prefixstringFalse
location_templatestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Function.Container

NameTypeDescriptionRequired
executableExecutableFalse
credentials_configurationFunction.CredentialsConfigurationFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Function.Container.Byte

NameTypeDescriptionRequired
containerFunction.ContainerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Function.Container.Record

NameTypeDescriptionRequired
containerFunction.ContainerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Function.Credentials

NameTypeDescriptionRequired
secretstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Function.CredentialsConfiguration

NameTypeDescriptionRequired
idCredentials.IdFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Gcp.BigQuery

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Gcp.BigQuery.Container

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
projectstringFalse
dataset_prefixstringFalse
location_templatestringFalse
partition_patternPatternFalse
staging_containerGcp.Gcs.ContainerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Gcp.Credentials

NameTypeDescriptionRequired
json_keystringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Gcp.Gcs

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Gcp.Gcs.Container

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
bucketstringFalse
prefixstringFalse
delimiterstringFalse
disable_random_location_infixbooleanFalse
location_templatestringFalse
location_suffixstringFalse
manifestManifestFalse
write_part_filesbooleanFalse
use_flat_part_file_layoutbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Immediate.Container

NameTypeDescriptionRequired
content_somebyteFalse
content_noneFalse
objectList<Immediate.Object>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Immediate.Object

NameTypeDescriptionRequired
namestringFalse
lengthint64False
content_shabyteFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Manifest

NameTypeDescriptionRequired
namestringFalse
formatManifest.FormatFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Manifest.Csv

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Manifest.Format

NameTypeDescriptionRequired
jsonManifest.JsonFalse
csvManifest.CsvFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Manifest.Json

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.MsSqlServer.Container

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
server_namestringFalse
portintegerFalse
use_encryptionbooleanFalse
database_namestringFalse
absAzure.Abs.ContainerFalse
location_templatestringFalse
update_policyUpdatePolicyFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.MsSqlServer.Credentials

NameTypeDescriptionRequired
sql_authenticationMsSqlServer.Credentials.SqlAuthenticationFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.MsSqlServer.Credentials.SqlAuthentication

NameTypeDescriptionRequired
userstringFalse
passwordstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Mysql

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Mysql.Container

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
hoststringFalse
portintegerFalse
databasestringFalse
location_templatestringFalse
disable_tlsbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Mysql.Credentials

NameTypeDescriptionRequired
usernamestringFalse
passwordstringFalse
certificateMysql.TlsCertificateFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Mysql.Location

NameTypeDescriptionRequired
containerMysql.ContainerFalse
table_namestringFalse
schemaMapFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Mysql.Partition.Container

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
hoststringFalse
portintegerFalse
databasestringFalse
tablestringFalse
location_templatestringFalse
disable_tlsbooleanFalse
sha_overridesContainer.ShaOverridesFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Mysql.TlsCertificate

NameTypeDescriptionRequired
server_castringFalse
client_certstringFalse
client_keystringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Snowflake.Container

NameTypeDescriptionRequired
credential_idCredentials.IdFalse
update_policyUpdatePolicyFalse
s3Aws.S3.ContainerFalse
gcsGcp.Gcs.ContainerFalse
absAzure.Abs.ContainerFalse
location_templatestringFalse
accountstringFalse
warehousestringFalse
databasestringFalse
snowflake_schemastringFalse
rolestringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Snowflake.Credentials

NameTypeDescriptionRequired
usernamestringFalse
passwordstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Spark.Container

NameTypeDescriptionRequired
storeSpark.StoreFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.io.io_pb2.Spark.Store

NameTypeDescriptionRequired
s3Aws.S3.ContainerFalse
gcsGcp.Gcs.ContainerFalse
absAzure.Abs.ContainerFalse
formatFormatFalse
content_encodingContentEncodingFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Granularity

Granularity of temporal partitioning, e.g., when doing partial
reductions.

NameTypeDescriptionRequired
timeGranularity.TimeFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Granularity.Time

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Granularity.Time.Auto

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Granularity.Time.Day

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Granularity.Time.Hour

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Granularity.Time.Minute

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Granularity.Time.Month

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Granularity.Time.Year

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.InputInfo

NameTypeDescriptionRequired
indexintegerFalse
uuidstringFalse
typestringFalse
idstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.InstanceType

NameTypeDescriptionRequired
smallInstanceType.SmallFalse
mediumInstanceType.MediumFalse
largeInstanceType.LargeFalse
xlargeInstanceType.XLargeFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.InstanceType.Large

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.InstanceType.Medium

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.InstanceType.Small

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.InstanceType.XLarge

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Operator

NameTypeDescriptionRequired
avro_formatterAvro.FormatterFalse
avro_parserAvro.ParserFalse
grok_parserGrok.ParserFalse
json_formatterJson.FormatterFalse
json_parserJson.ParserFalse
lambda_formatterLambda.FormatterFalse
lambda_parserLambda.ParserFalse
orc_formatterOrc.FormatterFalse
parquet_formatterParquet.FormatterFalse
parquet_parserParquet.ParserFalse
xsv_formatterXsv.FormatterFalse
xsv_parserXsv.ParserFalse
spark_functionSpark.FunctionFalse
sql_querySql.QueryFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.RecursionSpecification

NameTypeDescriptionRequired
seed_data_indexintegerFalse
retention_policyRecursionSpecification.RetentionPolicyFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.RecursionSpecification.RetentionPolicy

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.RecursionSpecification.RetentionPolicy.All

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.RecursionSpecification.RetentionPolicy.LastN

NameTypeDescriptionRequired
nintegerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Reduction

NameTypeDescriptionRequired
fullReduction.FullFalse
no_reductionReduction.NoReductionFalse
partialReduction.PartialFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Reduction.Full

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Reduction.NoReduction

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Reduction.Partial

NameTypeDescriptionRequired
partition_byobjectFalse
recursion_specificationRecursionSpecificationFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Reduction.Partial.PartitionSpecification

NameTypeDescriptionRequired
columnstringFalse
granularityGranularityFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Spark.Filter

NameTypeDescriptionRequired
functionSpark.Filter.FunctionFalse
compositeSpark.Filter.CompositeFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Spark.Filter.Composite

NameTypeDescriptionRequired
operatorSpark.Filter.Composite.OperatorFalse
filtersList<Spark.Filter>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Spark.Filter.Composite.Operator

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Spark.Filter.Function

NameTypeDescriptionRequired
namestringFalse
argsListFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Spark.Function

NameTypeDescriptionRequired
executableExecutableFalse
reductionReductionFalse
credentials_configurationFunction.CredentialsConfigurationFalse
job_configurationSpark.JobConfigurationFalse
additional_input_configsobjectFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Spark.Function.AdditionalInputConfig

NameTypeDescriptionRequired
required_metadata_columnsListFalse
filterSpark.FilterFalse
unioned_inputsList<InputInfo>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Spark.JobConfiguration

NameTypeDescriptionRequired
executor_instancesintegerFalse
executor_instance_typeInstanceTypeFalse
driver_instance_typeInstanceTypeFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.operator.operator_pb2.Sql.Query

NameTypeDescriptionRequired
sqlstringFalse
spark_job_configurationSpark.JobConfigurationFalse
override_fingerprintstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.pattern.pattern_pb2.Pattern

NameTypeDescriptionRequired
globstringFalse
regexstringFalse
exact_matchstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.preview.preview_pb2.Response

NameTypeDescriptionRequired
text_responseResponse.TextResponseFalse
records_responseResponse.RecordsResponseFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.preview.preview_pb2.Response.RecordsResponse

NameTypeDescriptionRequired
component_idIdFalse
recordsRecordsReadFalse
schemaMapFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.preview.preview_pb2.Response.TextResponse

NameTypeDescriptionRequired
lineListFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Array

NameTypeDescriptionRequired
element_schemaSchemaFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Binary

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Boolean

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Boolean.Parser

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Byte

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Char

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Date

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Date.Parser

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Decimal

NameTypeDescriptionRequired
precisionintegerFalse
scaleintegerFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Dictionary

NameTypeDescriptionRequired
key_schemaSchemaFalse
value_schemaSchemaFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Double

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Double.Parser

NameTypeDescriptionRequired
decimal_separatorstringFalse
thousands_separatorstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Field

NameTypeDescriptionRequired
namestringFalse
schemaSchemaFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Float

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Float.Parser

NameTypeDescriptionRequired
decimal_separatorstringFalse
thousands_separatorstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Int

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Int.Parser

NameTypeDescriptionRequired
thousands_separatorstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Long

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Long.Parser

NameTypeDescriptionRequired
thousands_separatorstringFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Map

NameTypeDescriptionRequired
fieldList<Field>False

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Nothing

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Schema

NameTypeDescriptionRequired
arrayArrayFalse
booleanBooleanFalse
byteByteFalse
charCharFalse
dateDateFalse
doubleDoubleFalse
floatFloatFalse
intIntFalse
longLongFalse
mapMapFalse
nothingNothingFalse
stringStringFalse
timestampTimestampFalse
shortShortFalse
decimalDecimalFalse
binaryBinaryFalse
dictionaryDictionaryFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Short

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.String

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.String.Parser

NameTypeDescriptionRequired
trimbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Timestamp

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.Timestamp.Parser

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.UpdatePolicy

NameTypeDescriptionRequired
errorUpdatePolicy.ErrorFalse
drop_tableUpdatePolicy.DropTableFalse
alter_tableUpdatePolicy.AlterTableFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.UpdatePolicy.AlterTable

NameTypeDescriptionRequired
alter_columns_on_type_mismatchbooleanFalse
drop_unknown_columnsbooleanFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.UpdatePolicy.DropTable

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.schema.schema_pb2.UpdatePolicy.Error

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding

NameTypeDescriptionRequired
crLineEnding.CrFalse
cr_lfLineEnding.CrLfFalse
lfLineEnding.LfFalse
lf_crLineEnding.LfCrFalse
ffLineEnding.FfFalse
lsLineEnding.LsFalse
nelLineEnding.NelFalse
psLineEnding.PsFalse
vtLineEnding.VtFalse

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.Cr

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.CrLf

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.Ff

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.Lf

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.LfCr

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.Ls

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.Nel

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.Ps

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.text.text_pb2.LineEnding.Vt

Object has no properties

[Back to top] [Back to API list] [Back to Models]

ascend.protos.worker.worker_pb2.RecordsRead

NameTypeDescriptionRequired
valueList<Struct>False

[Back to top] [Back to API list] [Back to Models]

Updated 2 days ago


Python SDK


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.