libiio-v1: Setting buffer length in mmap/dmabuf cases

[schwesinger]

Hi,

when creating IIO blocks (during creation of an IIO stream [1]), block_size of the IIO buffer is set to the size of the IIO block in case the mmap/dmabuf APIs are not used [2]. Later when the IIO buffer is enabled, the length attribute of the buffer is set to the number of samples derived from the block_size and sample_size [3] [4] [5].
When the dmabuf API is used with an IIO buffer block_size is not set and the length attribute of the buffer is left at its default value, which was 2048 samples (per channel) in my experiments.

In the FPGA design I'm using, the length attribute of the buffer is used to determine the number of samples that are to be transmitted per DMA transaction. For IIO blocks that can hold more than 2048 samples, the remaining block space is filled with "bogus" data*, the 2049th sample will be present in the next IIO block. This was observed when correlating the raw received data with data obtained from an ILA triggering on TLAST of the DMA FIFO core.

Manually setting buffer/length to the IIO block size via the sysfs interface fixes the issue and continuous transfers work as expected. Unfortunately, I was not able to programmatically set buffer/length using libiio: Iterating over the attributes of the IIO buffer in question only returns data_available, direction and length_align_bytes. Thus, I resorted to patching libiio to set the block_size of the IIO buffer during block creation as in the non-dmabuf case:

diff --git a/block.c b/block.c
index d26314fb..cab112ae 100644
--- a/block.c
+++ b/block.c
@@ -67,8 +67,6 @@ iio_buffer_create_block(struct iio_buffer *buf, size_t size)

                if (size > buf->length)
                      buf->length = size;
-
-               buf->block_size = size;
        }

        block->buffer = buf;
@@ -76,6 +74,7 @@ iio_buffer_create_block(struct iio_buffer *buf, size_t size)

        iio_mutex_lock(buf->lock);
        buf->nb_blocks++;
+       buf->block_size = size;
        iio_mutex_unlock(buf->lock);

        return block;
diff --git a/local.c b/local.c
index ca5edfc5..e8065133 100644
--- a/local.c
+++ b/local.c
@@ -328,9 +328,6 @@ static int local_enable_buffer(struct iio_buffer_pdata *pdata,
 {
        int ret;

-       if ((pdata->dmabuf_supported | pdata->mmap_supported) != !nb_samples)
-               return -EINVAL;
-
        if (enable && nb_samples) {
                ret = local_set_buffer_size(pdata, nb_samples);
                if (ret)

However, I'm absolutely unsure, if this is the right approach because the checks in local_enable_buffer, can be interpreted as deliberate checks for an IIO buffer block_size set to zero (converted to number of samples nb_samples). As I don't know all the other use cases/contexts for using IIO blocks, this patch could well introduce some unintended side effects. So could anybody more familiar with libiio internals have a look?

*) For testing purposes I initialized the mmap()ed dmabuf memory in local_create_dmabuf() with a recurring 4-byte pattern. And repeated my test without setting the IIO buffer length. The pattern was found in the transmitted data with series of zeroes interspersed. So maybe it's advisable to zero dmabuf memory right after allocation?

[rgetz]

Ok - tracing the current implementation, from the public/top level API:

libiio/buffer.c : iio_device_create_buffer()

struct iio_buffer *iio_device_create_buffer(
        struct iio_device *dev,
        size_t samples, bool cyclic)
{
    ...
    ret = buf->backend->enable(pdata, true, samples);

Passes samples from the public API. The enable() function is backend-specific. In the local backend, this is local_enable_buffer().

libiio/local.c : local_enable_buffer()

static int local_enable_buffer(struct iio_buffer_pdata *pdata,
                               bool enable, size_t nb_samples)
{
    ...
    // 🚫 GATE IS HERE
    if ((pdata->dmabuf_supported | pdata->mmap_supported) != !nb_samples)
        return -EINVAL;

This expression evaluates as:

if ((dmabuf OR mmap) is enabled) != (nb_samples == 0)
   return error

This is a deliberate gate to prevent setting buffer/length when using dmabuf or mmap, as a protective guardrail — it’s there to Protect Users from Misconfiguring the Hardware : When dmabuf or mmap is used, buffer length and memory are tightly controlled by the kernel or hardware driver. The libiio user should not assume they can freely override buffer/length via userspace.

Why?

• If Users Set Length in dmabuf Mode:
  • They may set a length the hardware can't handle (e.g., not aligned to FIFO burst size or DMA descriptor limitations).
  • It may conflict with preallocated buffer sizes in the kernel, leading to overruns or underruns.
  • Hardware (like FPGA DMA controllers) might expect fixed-size transactions, and overriding the length causes incomplete or malformed transfers.
  • Drivers may not validate user-supplied lengths against hardware capabilities — leading to silent corruption.

So Why Allow Length Override When dmabuf Is Not Used? In the non-dmabuf path:

• libiio owns the memory
• It allocates and manages the buffer itself
• So it can safely interpret and propagate samples to the sysfs buffer/length

From the perspective of kernel and driver developers:
If you're using dmabuf, we assume you're using the kernel driver as designed. Don't override internals unless you really know what you're doing.

I think that is the core part of the question - how do we let experienced users explicitly override buffer/length when using dmabuf, without breaking safety for everyone else? I can think of three things:

1. Environment Variable Override : change gate to:

const char *force_length = getenv("LIBIIO_FORCE_BUFFER_LENGTH");

if ((pdata->dmabuf_supported || pdata->mmap_supported) && nb_samples && !force_length)
    return -EINVAL;

2. add libiio Private API / Attribute for buffer length.
3. use using the context-level attributes : io_context_set_attr("buffer_force_length", "true")

I'm not sure I like 1 - although it's non-invasive, and has no API change - its not visible in the C API, and too easy to forget to set in production environments.
Number 2 isn't much better - while it is controlled and programmable, it clutters the API, and people will use it without understanding it's an advanced maneuver - and it will just cause problems.
Which leaves 3 - it's similar in spirit to environment variable, but integrated in API (which is better), but off to the side so that most developers will not see it.

Would be interested in @dNechita and @nunojsa or @pcercuei (if he has time) think...

[dNechita]

It is practical to have this setting per buffer. Therefore, options 1 and 3 aren't that great. The biggest issue with option 3 is that the context attributes are read-only. Writing to a context attribute has no effect of any type.
I suggest adding bool force_length to the struct struct iio_buffer_params.

[nunojsa]

So, I'm not so sure this is an actual issue or something we should address. If I'm not mistaken, when we're using the high speed interface buffer/length is not used at all (taken into account) 1. It's only relevant when we are using the slow fileio based approach. So even if we add any mechanism for writing that file, that pretty much means nothing in terms of DMA mappings or dmabufs allocation.

Manually setting buffer/length to the IIO block size via the sysfs interface fixes the issue and continuous transfers work as expected

That's because you're assuming that's being used at all. So I think you can very well make things work without buffer/length... The way it is supposed to work with dmabufs is that each dmabuf will be a multiple of your sample size and when you create your stream 2 what you pass is the number of samples you want per dmabuf. If you allocate the block directly look at 2 to see how the block size relates to sample_size * samples_count.

libiio handles things and automatically writes that file when not in high speed mode.

So, unless I'm missing something I think this is a non issue...

[schwesinger]

Thanks for all your replies!

The core issue I'm facing is, that unless the size of a single DMA transaction/packet matches the used IIO block size, the data stream gets corrupted.

On the FPGA-side we have a custom IP core that preprocesses ADC samples and is connected to a DMA core (not the AD AXI DMA) through an AXI stream. The important thing here is, that the DMA packet size is determined by the TLAST signal as part of the AXI stream protocol. This TLAST signal is generated in our custom IP core (controlled by a register) and thus, is external to the DMA core itself.

When using libiio's high-level streaming API the IIO block size is varying depending on sample size, number of active channels and requested number of samples (per block/DMABUF). Iterating from a non-highspeed use-case I falsely assumed that, on the IIO driver side, buffer/length would hold the IIO block size (populated from libiio) and thus could be used to "automatically" communicate that bit of information to the FPGA side via a iio_buffer_setup_ops->preenable callback.

Instead of relying on buffer/length for this, a distinct IIO device attribute could be introduced in our IIO driver that gets set from the userspace application as soon as the IIO block size is available. However, this feels quite fragile, because now the userspace app. is responsible for setting the right value and implementing safeguards in the IIO driver is non-trivial/hard/impossible because IIO block size is user supplied (active channel selection) and use-case dependent (sample size may change with different boards+ADCs).

Are there any better options to solve this?

From a very brief look I saw the AD AXI DMAC also supports the AXI stream interface. But in none of the example drivers I looked at I could find anything that looks like propagating IIO block sizes to the HW. What am I missing here?

---

So yes, this is probably not a libiio issue but more a general design issue. Let me know if there's a better place to further discuss these things.

[rgetz]

On the FPGA-side we have a custom IP core that preprocesses ADC samples and is connected to a DMA core (not the AD AXI DMA) through an AXI stream.

I think you are answering your own question...

Feature	Xilinx AXI DMA (S2MM)	ADI AXI DMAC (Stream to Memory)
Who generates TLAST?	✅ Your upstream IP (source)	❌ DMA core itself generates TLAST
Who determines packet size?	Your logic (via register or FSM)	The DMA core (via transfer_size register)
Stream segmenting	Must match TLAST to buffer size	DMA automatically ends stream after N bytes
Design complexity	Higher – must coordinate hardware & SW	Lower – software-controlled via IIO driver
TLAST input to DMA	✔️ Required	❌ Not required (generated internally)

The ADI AXI DMAC core supports two primary modes for stream-to-memory (S2MM) - this seems to be what you are suggesting:

Mode	Description
Fixed-length transfers	DMA generates TLAST after TRANSFER_SIZE bytes (most common in ADC chains)
Variable-length transfers	Upstream IP drives TLAST, DMA ends transfer when TLAST is seen

This is controlled internally by the DMAC configuration/state logic. If TLAST is present and valid, the core will terminate the current buffer write and issue an interrupt or flag to the kernel driver. So the ADI DMA core can terminate either:

• When it sees TLAST, or
• When it reaches the transfer size

I know we did that in that for the QPSK modem design - but that ultimately did not use libiio for that. I don't know if there is a different example that does use variable-length transfers (TLAST asserted by the IP) with an IIO device and libiio.

again maybe @nunojsa or @acostina might know.

-Robin

[schwesinger]

Thanks, that's what I figured.

I know we did that in that for the QPSK modem design - but that ultimately did not use libiio for that. I don't know if there is a different example that does use variable-length transfers (TLAST asserted by the IP) with an IIO device and libiio.

Alright, I just wanted to make sure I'm not missing something obvious. I think this issue can be closed then. Still, I'd be interested in looking at some IIO driver + libiio code implementing variable length transfers, if there's any.