Skip to content

v0.16

Latest
Latest
Compare
Choose a tag to compare
@github-actions github-actions released this 05 Aug 08:00
· 2 commits to main since this release
v0.16
6ba5f6b

What's New

Array Support (Preview)

This release introduces support for multi-dimensional arrays, enabling you to stream array data from Kafka to QuestDB.
Important: Array support is currently in preview mode and subject to change in future releases!

Supported Array Types

  • 1D, 2D, and 3D numeric arrays
  • Both schema-based and schema-free messages
  • Nested arrays within complex structures

Example

{
  "symbol": "AAPL",
  "ohlc_5min": [[150.1, 151.2, 149.8, 150.9], [150.9, 152.1, 150.3, 151.5]],
  "volume": [12500, 13200]
}

Current Limitations

  • Not supported: String arrays, mixed-type arrays, null elements, empty arrays
  • Future: Support for up to 32 dimensions and empty arrays.

Array Type Constraints

QuestDB Server Limitation: QuestDB currently supports arrays of floating-point numbers only.

Schema-less Records

  • Arrays of any numeric type are automatically converted to arrays of doubles
  • No errors will occur as all numbers are sent as floating-point

Schema-based Records

  • The schema must specify floating-point types for arrays
  • Arrays with fixed-point (integer) schemas will throw an error
  • Make sure your schemas use FLOAT32 or FLOAT64 for array elements

Multidimensional Array Requirements

QuestDB Server Limitation: Multidimensional arrays must have uniform dimensions (non-jagged).

Examples:

✅ Supported - uniform 2D array (2x3)
{
  "matrix": [
    [1.0, 2.0, 3.0], 
    [4.0, 5.0, 6.0]
  ]
}

❌ Not supported - jagged array (different row lengths)
{
  "matrix": [
    [1.0, 2.0],
    [3.0, 4.0, 5.0]
  ]
}

✅ Supported - uniform 3D array (2x2x2)
{
  "tensor": [
    [[1.0, 2.0], [3.0, 4.0]], 
    [[5.0, 6.0], [7.0, 8.0]]
  ]
}

Messages containing jagged arrays will be rejected by the connector.
Read more about arrays in QuestDB: https://github.com/questdb/kafka-questdb-connector/releases

Upgrading from v0.15

This release is backward compatible with v0.15. Simply update your connector version to start using array support.

Full Changelog: v0.15...v0.16

Assets 3
Loading