connectors

Creates, updates, deletes, gets or lists a connectors resource.

Overview

Name	connectors
Type	Resource
Id	confluent.connect.connectors

Fields

Name	Datatype	Description
name	string	Name of the connector
config	object	Configuration parameters for the connector. These configurations are the minimum set of key-value pairs (KVP) which can be used to define how the connector connects Kafka to the external system. Some of these KVPs are common to all the connectors, such as connection parameters to Kafka, connector metadata, etc. The list of common connector configurations is as follows - cloud.environment - cloud.provider - connector.class - kafka.api.key - kafka.api.secret - kafka.endpoint - kafka.region - name A specific connector such as GcsSink would have additional parameters such as gcs.bucket.name, flush.size, etc.
tasks	array	List of active tasks generated by the connector
type	string	Type of connector, sink or source

Methods

Name	Accessible by	Required Params	Description
list_connectv1connectors	SELECT	environment_id, kafka_cluster_id	Retrieve a list of "names" of the active connectors. You can then make a read request for a specific connector by name.
read_connectv1connector	SELECT	connector_name, environment_id, kafka_cluster_id	Get information about the connector.
create_connectv1connector	INSERT	environment_id, kafka_cluster_id	Create a new connector. Returns the new connector information if successful.
delete_connectv1connector	DELETE	connector_name, environment_id, kafka_cluster_id	Delete a connector. Halts all tasks and deletes the connector configuration.
pause_connectv1connector	EXEC	connector_name, environment_id, kafka_cluster_id	Pause the connector and its tasks. Stops message processing until the connector is resumed. This call is asynchronous and the tasks will not transition to PAUSED state at the same time.
resume_connectv1connector	EXEC	connector_name, environment_id, kafka_cluster_id	Resume a paused connector or do nothing if the connector is not paused. This call is asynchronous and the tasks will not transition to RUNNING state at the same time.

SELECT examples

Retrieve a list of "names" of the active connectors. You can then make a read request for a specific connector by name.

SELECT
name,
config,
tasks,
type
FROM confluent.connect.connectors
WHERE environment_id = '{{ environment_id }}'
AND kafka_cluster_id = '{{ kafka_cluster_id }}';

INSERT example

Use the following StackQL query and manifest file to create a new connectors resource.

Required Properties

/*+ create */
INSERT INTO confluent.connect.connectors (
data__name,
data__config,
data__offsets,
environment_id,
kafka_cluster_id
)
SELECT 
'{{ name }}',
'{{ config }}',
'{{ offsets }}',
'{{ environment_id }}',
'{{ kafka_cluster_id }}'
;

All Properties

/*+ create */
INSERT INTO confluent.connect.connectors (
environment_id,
kafka_cluster_id
)
SELECT 
'{{ environment_id }}',
'{{ kafka_cluster_id }}'
;

Manifest

- name: connectors
  props:
    - name: environment_id
      value: string
    - name: kafka_cluster_id
      value: string
    - name: name
      value: string
    - name: config
      props:
        - name: connector.class
          value: string
        - name: name
          value: string
        - name: kafka.api.key
          value: string
        - name: kafka.api.secret
          value: string
        - name: confluent.connector.type
          value: string
        - name: confluent.custom.plugin.id
          value: string
        - name: confluent.custom.connection.endpoints
          value: string
        - name: confluent.custom.schema.registry.auto
          value: string
    - name: offsets
      value: array
      props:
        - name: partition
          value: object
        - name: offset
          value: object

DELETE example

Deletes the specified connectors resource.

/*+ delete */
DELETE FROM confluent.connect.connectors
WHERE connector_name = '{{ connector_name }}'
AND environment_id = '{{ environment_id }}'
AND kafka_cluster_id = '{{ kafka_cluster_id }}';