Generate
Generate a Connectorโ
CDK generate helps developers build a sample Connector project by answering a few simple questions.
Use cdk generate
to create a new connector project:
$ cdk generate
๐คท Project Name: my-connector
๐คท Please set a group name: acme
๐คท Which type of Connector would you like [source/sink]? ยท source
๐คท Will your Connector be public? ยท false
[1/8] Done: .gitignore
[2/8] Done: Cargo.toml
[3/8] Done: Connector.toml
[4/8] Done: README.md
[5/8] Done: sample-config.yaml
[6/8] Done: src/config.rs
[7/8] Done: src/main.rs
[8/8] Done: src
The generator created Rust project ready to compile:
$ tree
.
โโโ Cargo.toml
โโโ Connector.toml
โโโ README.md
โโโ sample-config.yaml
โโโ src
โโโ config.rs
โโโ main.rs
This a simple connector with the code in src/main.rs
:
mod config;
use config::CustomConfig;
use fluvio::{RecordKey, TopicProducerPool};
use fluvio_connector_common::{
connector,
Result
};
#[connector(source)]
async fn start(config: CustomConfig, producer: TopicProducerPool) -> Result<()> {
println!("Starting my-connector source connector with {config:?}");
for i in 1..1000 {
let value = format!("Hello, Fluvio - {i}");
producer.send(RecordKey::NULL, value).await?;
producer.flush().await?;
std::thread::sleep(std::time::Duration::from_millis(1000));
}
Ok(())
}
Connectors may also have define configuration parameters as shown in src/config.rs
:
use fluvio_connector_common::connector;
#[connector(config)]
#[derive(Debug)]
pub(crate) struct CustomConfig {
#[allow(dead_code)]
foo: String,
}
The Connector.toml
file contains the definition of the Connector parameters required to load the file in the Cluster and publish it to Connector Hub.
[package]
name = "my-connector"
group = "acme"
version = "0.1.0"
apiVersion = "0.1.0"
fluvio = "0.10.0"
description = ""
license = "Apache-2.0"
visibility = "private"
[direction]
source = true
[deployment]
binary = "my-connector"
[custom.properties.foo]
title = "Foo"
description = "Foo"
type = "string"
Sectionsโ
package
is used to build the connector FQDNacme/my-connector@0.1.0
, and the description to publish to Hub. Thegroup
name is equivalent to the package owner in the Hub.direction
is used to declare the direction data flows through the connector, with respect to the Fluvio cluster. An inbound connector usessource = true
, and and outbound connectdor usessink = true
custom.properties.foo
defines a user configuration keyfoo
that can be used in the logic of the connector.
The project is ready to build and test.