From 5ed9cee4f564b7364e3538799f0352c6cb5bc14b Mon Sep 17 00:00:00 2001 From: Jaromir Hamala Date: Mon, 28 Jul 2025 11:17:51 +0200 Subject: [PATCH 1/2] document ${partition} variable also: changed default types from string to varchar since that's what questdb now uses. --- documentation/third-party-tools/kafka.md | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/documentation/third-party-tools/kafka.md b/documentation/third-party-tools/kafka.md index 3c5ddb56a..a69c6a452 100644 --- a/documentation/third-party-tools/kafka.md +++ b/documentation/third-party-tools/kafka.md @@ -287,9 +287,9 @@ Example: Consider the following Kafka message: The connector will create a table with the following columns: -| firstname string | lastname string | age long | address_street string | address_city string | -| --------------------------- | -------------------------- | ------------------- | -------------------------------- | ------------------------------ | -| John | Doe | 30 | Main Street | New York | +| firstname varchar | lastname varchar | age long | address_street varchar | address_city varchar | +|------------------------------|-----------------------------|---------------------|-----------------------------------|---------------------------------| +| John | Doe | 30 | Main Street | New York | ### Client configuration string @@ -568,9 +568,10 @@ table=example_table [...] ``` -The `table` options supports simple templating. You can use the following -variables in the table name: `${topic}`, `${key}`. The connector will replace -these variables with the actual topic name and key value from the Kafka message. +The table options supports simple templating. You can use the following +variables in the table name: `${topic}`, `${key}`, `${partition}`. The connector will replace +these variables with the actual topic name, key value, and partition number from the Kafka message. + Example: @@ -579,14 +580,15 @@ name=questdb-sink connector.class=io.questdb.kafka.QuestDBSinkConnector client.conf.string=http::addr=localhost:9000; topics=example-topic -table=from_kafka_${topic}_${key}_suffix +table=from_kafka_${topic}_${key}_${partition}_suffix [...] ``` The placeholder `${key}` will be replaced with the actual key value from the Kafka message. If the key is not present in the message, the placeholder will be -replaced with the string `null`. +replaced with the string `null`. The placeholder `${partition}` will be replaced +with the Kafka partition number from which the message originates. #### Table schema From f278d35036de07a0fe96cd1047fd94d0069798eb Mon Sep 17 00:00:00 2001 From: Jaromir Hamala Date: Mon, 28 Jul 2025 11:20:52 +0200 Subject: [PATCH 2/2] tweaks --- documentation/third-party-tools/kafka.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/third-party-tools/kafka.md b/documentation/third-party-tools/kafka.md index a69c6a452..1cb1bcfe1 100644 --- a/documentation/third-party-tools/kafka.md +++ b/documentation/third-party-tools/kafka.md @@ -568,7 +568,7 @@ table=example_table [...] ``` -The table options supports simple templating. You can use the following +The `table` option supports simple templating. You can use the following variables in the table name: `${topic}`, `${key}`, `${partition}`. The connector will replace these variables with the actual topic name, key value, and partition number from the Kafka message.