Heap settings using malloc

[maaikez]

Hello,

In our project, we use malloc (/calloc) to allocate memory. We have 16MB available memory, which should be plenty enough for what we do.

However, some time ago we started using hostap (CONFIG_WIFI_NM_WPA_SUPPLICANT also enables COMMON_LIBC_MALLOC) and had to enable CONFIG_HEAP_MEM_POOL_SIZE and since it uses mbedtls, also CONFIG_MBEDTLS_ENABLE_HEAP and CONFIG_MBEDTLS_HEAP_SIZE. We would like to keep the CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE to -1, but the application crashes in that case:

***** BUS FAULT *****
Imprecise data bus error
...
ZEPHYR FATAL ERROR 26: Unknown error on CPU 0

mostly in the logging thread.

If I understand correctly, CONFIG_HEAP_MEM_POOL_SIZE is for k_malloc, k_free, k_heap*, etc.
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE is for malloc, free, etc.
Is that correct?

So I am not sure what to set for CONFIG_HEAP_MEM_POOL_SIZE, is that only for the modules we use or also for Zephyr?

And if I do set CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE to a value, I can not even set it to a value higher than a few hundred kB, because it will crash (kernel panic):

ASSERTION FAIL [left_chunk(h, right_chunk(h, c)) == c] @ WEST_TOPDIR/zephyr/lib/heap/heap.c:183
        corrupted heap bounds (buffer overflow?) for memory at 0x804b0f20

So I am a bit confused how this works and what values I should use for the heap sizes. Maybe I am missing some config values? I tried setting CONFIG_NEWLIB_LIBC_MAX_MAPPED_REGION_SIZE but that does not help, but I am only guessing now instead of understanding the problem.

I think nicest would be to set CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE to -1, so the application can use all the remaining heap.

And is there a way in the API to find out how much heap is used? I found sys_heap_runtime_stats_get but it is only having a total value of just above 97000.

So my questions are:

• What are the reasons it is not possible to set CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE to -1?
• Why can I not set CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE to a value of a few MB?
• Are there other config items I need for above config items to work?
• And actually a bit basic question: How does it work with the heaps in Zephyr anyway? (I did read https://docs.zephyrproject.org/latest/kernel/memory_management/heap.html but it doesn't explain much about the libc heap).

I hope someone can explain a bit more and help out. Thanks in advance!

[ldomaigne]

As far as I can see, that will depend on the C library you are using. Could you please check with west build -t menuconfig and look for the symbol ending with LIBC? (press / and give the string: LIBC$).

In particular, PICOLIBC will be setting CONFIG_COMMON_LIBC_MALLOC, which relies on the kernel heap. And therefore CONFIG_HEAP_MEMPOOL_SIZE is the arena size to use for the alloc functions.

Not sure if that helps... I'm also somewhat confused by the documentation.

[maaikez]

Currently Newlib is used (I think it was default). Is picolib better? And what to use for newlib?

[ldomaigne]

picolibc is the default since Zephyr 3.5.

[maaikez]

With Picolibc I do not have those heap issues and I can just leave out CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE (so default value is -1). So I am wondering what settings I should use for newlib so it can work.

If picolib works that's also fine, but I can not seem to enable CONFIG_PICOLIBC_IO_FLOAT, which is a problem.

[ldomaigne]

If picolib works that's also fine, but I can not seem to enable CONFIG_PICOLIBC_IO_FLOAT, which is a problem.

Why not? If certain options are not available, like CONFIG_PICOLIBC_IO_FLOAT=y, you should investigate what is the cause. I know, it's easier said than done... You can either look at the output of the build. During the config header file generation (autoconf.h), the build system will warn if some options were invalidated. The other way is to use menuconfig or guiconfig, and check why the the config option is OFF.

So I am wondering what settings I should use for newlib so it can work.

This is explain here, which is essentially an English translation of this code:

zephyr/lib/libc/newlib/libc-hooks.c

Lines 39 to 117 in e22ca6b

	#ifndef CONFIG_NEWLIB_LIBC_CUSTOM_SBRK
	#define LIBC_BSS K_APP_BMEM(z_libc_partition)
	#define LIBC_DATA K_APP_DMEM(z_libc_partition)
	/*
	* End result of this thorny set of ifdefs is to define:
	*
	* - HEAP_BASE base address of the heap arena
	* - MAX_HEAP_SIZE size of the heap arena
	*/
	#ifdef CONFIG_MMU
	#ifdef CONFIG_USERSPACE
	struct k_mem_partition z_malloc_partition;
	#endif
	LIBC_BSS static unsigned char *heap_base;
	LIBC_BSS static size_t max_heap_size;
	#define HEAP_BASE heap_base
	#define MAX_HEAP_SIZE max_heap_size
	#define USE_MALLOC_PREPARE 1
	#elif CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE
	/* Arena size expressed in Kconfig, due to power-of-two size/align
	* requirements of certain MPUs.
	*
	* We use an automatic memory partition instead of setting this up
	* in malloc_prepare().
	*/
	K_APPMEM_PARTITION_DEFINE(z_malloc_partition);
	#define MALLOC_BSS K_APP_BMEM(z_malloc_partition)
	/* Compiler will throw an error if the provided value isn't a
	* power of two
	*/
	MALLOC_BSS static unsigned char
	__aligned(CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE)
	heap_base[CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE];
	#define MAX_HEAP_SIZE CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE
	#define HEAP_BASE heap_base
	#else /* Not MMU or CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE */
	#define USED_RAM_END_ADDR POINTER_TO_UINT(&_end)
	#ifdef Z_MALLOC_PARTITION_EXISTS
	/* Start of malloc arena needs to be aligned per MPU
	* requirements
	*/
	struct k_mem_partition z_malloc_partition;
	#if defined(CONFIG_ARM) || defined(CONFIG_ARM64)
	#define HEAP_BASE ROUND_UP(USED_RAM_END_ADDR, \
	CONFIG_ARM_MPU_REGION_MIN_ALIGN_AND_SIZE)
	#elif defined(CONFIG_ARC)
	#define HEAP_BASE ROUND_UP(USED_RAM_END_ADDR, \
	Z_ARC_MPU_ALIGN)
	#elif defined(CONFIG_XTENSA)
	#define HEAP_BASE ROUND_UP(USED_RAM_END_ADDR, \
	XCHAL_MPU_ALIGN)
	#else
	#error "Unsupported platform"
	#endif /* CONFIG_<arch> */
	#define USE_MALLOC_PREPARE 1
	#else
	/* End of kernel image */
	#define HEAP_BASE USED_RAM_END_ADDR
	#endif
	/* End of the malloc arena is the end of physical memory */
	#if defined(CONFIG_XTENSA)
	/* TODO: Why is xtensa a special case? */
	extern char _heap_sentry[];
	#define MAX_HEAP_SIZE (POINTER_TO_UINT(&_heap_sentry) - \
	HEAP_BASE)
	#else
	#define MAX_HEAP_SIZE (KB(CONFIG_SRAM_SIZE) - (HEAP_BASE - \
	CONFIG_SRAM_BASE_ADDRESS))
	#endif /* CONFIG_XTENSA */
	#endif

It depends if you have a) a MMU, b) a MPU and using userspace threads, c) nothing of that sort or d) use your own custom sbrk().

It's a bit challenging to guide you, as we might be missing some key elements. The easiest would be to have a minimal example that reproduces the problem you are facing (again, easier said than done).

[maaikez]

Yes, sorry for the limited information. I thought I was missing something with the configuration options for the heap with newlib. It's a bit complex to make an example indeed, because we ported hostap from zephyr 4 to zephyr 3, but we might have missed some things there.

The CONFIG_PICOLIBC_IO_FLOAT is not even there in menuconfig. I did not find out why yet, but I will definitely look into it.

Thanks for your help anyway, will try to investigate further!

[maaikez]

It turned out that COMMON_LIBC_MALLOC, that was automatically enabled by hostap, did not work together with newlib (which was already enabled in our project).