[oneTBB] Add a function to create a set of NUMA bound task arenas #650

isaevil · 2025-10-02T09:18:10Z

Add a page that describes helper APIs to simplify constrained arena creations. Currently it is a single function oneapi::tbb::create_numa_task_arenas from corresponding RFC.

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

isaevil · 2025-10-02T10:25:46Z

@aleksei-fedotov @akukanov Please take a look.

aleksei-fedotov · 2025-10-02T12:49:49Z

source/elements/oneTBB/source/constrained_arena_helpers.rst

+.. cpp:function:: std::vector<oneapi::tbb::task_arena> create_numa_task_arenas(oneapi::tbb::task_arena::constraints other_constraints = {}, unsigned reserved_slots = 0)
+
+    Returns a ``std::vector`` of ``task_arena`` objects, each bound to a separate NUMA node.
+    The number of created ``task_arena`` is equal to the number of NUMA nodes available on the system.


Shall it be less strict and, thus, more accurate?

Suggested change

The number of created ``task_arena`` is equal to the number of NUMA nodes available on the system.

The number of created ``task_arena`` is equal to the number of NUMA nodes detected on the system.

Should we require that this is the same set of nodes that is returned by tbb::info::numa_nodes()?

I think it is fair to require that as well, since the intent of this API is to provide shortcut to a common usage pattern of NUMA-constrained arenas.

Updated the description

source/elements/oneTBB/source/constrained_arena_helpers.rst

source/elements/oneTBB/source/nested-aux-interfaces.rst

source/elements/oneTBB/source/constrained_arena_helpers.rst

Co-authored-by: Aleksei Fedotov <fedotov.aleksei@gmail.com>

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

source/elements/oneTBB/source/task_scheduler/task_arena/task_arena_cls.rst

Co-authored-by: Alexey Kukanov <alexey.kukanov@intel.com>

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

Co-authored-by: Alexey Kukanov <alexey.kukanov@intel.com>

source/elements/oneTBB/source/task_scheduler/task_arena/task_arena_cls.rst

kboyarinov · 2025-10-23T14:28:35Z

source/elements/oneTBB/source/task_scheduler/task_arena/task_arena_cls.rst

+    the ``task_arena`` objects. The ``numa_id`` value in ``constraints`` is automatically
+    replaced with a separate value for each created arena.


The second sentence is not 100% clear for me.
As far as I understand, the numa_id value in constraints is ignored and the corresponding numa_id` from tbb::info::numa_nodes`` is used instead.

May be something like this will be easier to understand?

Suggested change

the ``task_arena`` objects. The ``numa_id`` value in ``constraints`` is automatically

replaced with a separate value for each created arena.

the ``task_arena`` objects. The ``numa_id`` value specified in ``constraints`` is disregarded, and instead,

the corresponding ``numa_id`` from ``tbb::info::numa_nodes()`` is used during construction

of each particular ``task_arena``.

Also, should we specify this somehow if the error occurs during the system parsing as part of create_numa_task_arenas()? Will the numa_id be ignored in this case?

I understand that if the error occurs in create_numa_task_arenas, then it is hard to imagine that there is a valid numa_id in constrains, but may be it still make sense to specify the behavior explicitly somehow.

Using another value instead of an ignored parameter and replacing that parameter with the other value is essentially the same thing. Either wording does not mandate a specific implementation. The proposed possible implementation (https://github.com/uxlfoundation/oneTBB/blob/master/rfcs/proposed/numa_support/create-numa-arenas.md#possible-implementation) uses the replacement, I would say.

I wanted to avoid words like "ignored" - especially without saying what is used instead. But I can see how this might still be confusing.

Maybe it is better to say that the numa_id parameter for constraints will be automatically set to a certain value:

Suggested change

the ``task_arena`` objects. The ``numa_id`` value in ``constraints`` is automatically

replaced with a separate value for each created arena.

the ``task_arena`` objects. For each created arena, the ``numa_id`` value in ``constraints``

is automatically set to the corresponding NUMA node ID from ``tbb::info::numa_nodes()``.

Applied suggestions from here and in the comment above from the different discussion.

Co-authored-by: Konstantin Boyarinov <konstantin.boyarinov@intel.com> Co-authored-by: Alexey Kukanov <alexey.kukanov@intel.com>

…scovery Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

akukanov

Looks good to me.

vossmjp · 2025-10-27T15:09:14Z

source/elements/oneTBB/source/task_scheduler/task_arena/task_arena_cls.rst

        }

-        for (int i = 0; i < numa_nodes.size(); i++) {
+        for (int i = 0; i < arenas.size(); i++) {


This pattern demonstrates the usage, but it would be better to execute in one task_arena after enqueuing in the others. Since none of the arenas reserve a slot for a master, the main thread might wait on a fully subscribed arena, and so cannot actively participate.

Remember, it's the specification. The examples here are intended to illustrate usage. In the developer guide, we have a better example: https://uxlfoundation.github.io/oneTBB/main/tbb_userguide/Guiding_Task_Scheduler_Execution.html#set-numa-node

aleksei-fedotov

Content looks good to me. Although, I would also tweak a little representational aspects.

aleksei-fedotov · 2025-10-28T10:28:32Z

source/elements/oneTBB/source/task_scheduler/task_arena/task_arena_cls.rst

Do we have limit on the line width? Proper alignment will help avoiding unnecessary horizontal scrollings and make the rendering neater.

Consider:

The HTML page looks like this:

Seems acceptable, and I am not sure if manual line break can do better in this case.

As far as I know, you can't make indentation when using .. cpp::function with line breaks, so at best it will end up like this:

std::vector<task_arena> create_numa_task_arenas( task_arena::constraints constraints = {}, int reserved_slots = 0)

akukanov · 2025-10-28T11:06:13Z

source/elements/oneTBB/source/task_scheduler/task_arena/task_arena_cls.rst

+            std::vector<task_arena> create_numa_task_arenas(task_arena::constraints constraints = {},
+                                                            unsigned reserved_slots = 0);


In other functions, similar parameters are named a bit different. Compare:

std::vector<task_arena> create_numa_task_arenas(task_arena::constraints constraints = {}, unsigned reserved_slots = 0)

void initialize(constraints a_constraints, unsigned reserved_for_masters = 1, priority a_priority = priority::normal)

I suggest to use reserved_slots and perhaps constraints_ (or maybe a_constraint?) consistently for all functions.

Should we also change a_priority to priority_ for consistency?

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

Add a page with constraints helpers to the spec

f819fc1

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

isaevil force-pushed the add-numa-arena-helper branch from 93dbd8a to f819fc1 Compare October 2, 2025 09:32

Add newline to notion

6caccd4

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

akukanov added the TBB label Oct 2, 2025

aleksei-fedotov reviewed Oct 2, 2025

View reviewed changes

akukanov reviewed Oct 17, 2025

View reviewed changes

source/elements/oneTBB/source/nested-aux-interfaces.rst Outdated Show resolved Hide resolved

source/elements/oneTBB/source/constrained_arena_helpers.rst Outdated Show resolved Hide resolved

source/elements/oneTBB/source/constrained_arena_helpers.rst Outdated Show resolved Hide resolved

isaevil and others added 3 commits October 20, 2025 16:37

Apply suggestions from code review

b9cc94d

Co-authored-by: Aleksei Fedotov <fedotov.aleksei@gmail.com>

Move as non-member function into task_arena spec, update example

43f749f

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

Remove constrained_arena_helpers.rst

2438028

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

isaevil requested review from akukanov and aleksei-fedotov October 21, 2025 10:43

Fix underline

532fbb4

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

akukanov reviewed Oct 22, 2025

View reviewed changes

akukanov changed the title ~~[oneTBB] Add a page with constraints helpers to the spec~~ [oneTBB] Add a function to create a set of NUMA bound task arenas Oct 22, 2025

isaevil and others added 4 commits October 23, 2025 11:19

Apply suggestions from code review

25159ee

Co-authored-by: Alexey Kukanov <alexey.kukanov@intel.com>

Fix example

94f1d5d

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

Move the sentence about error

c08478f

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

Apply suggestion about arguments

d2be07c

Co-authored-by: Alexey Kukanov <alexey.kukanov@intel.com>

akukanov requested review from kboyarinov and vossmjp October 23, 2025 10:35

kboyarinov reviewed Oct 23, 2025

View reviewed changes

isaevil and others added 2 commits October 24, 2025 16:32

Apply suggestions from code review

c16ef5a

Co-authored-by: Konstantin Boyarinov <konstantin.boyarinov@intel.com> Co-authored-by: Alexey Kukanov <alexey.kukanov@intel.com>

Clarify the parameters of task_arena in case of failed system info di…

b730cab

…scovery Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

akukanov approved these changes Oct 24, 2025

View reviewed changes

kboyarinov approved these changes Oct 24, 2025

View reviewed changes

vossmjp reviewed Oct 27, 2025

View reviewed changes

aleksei-fedotov approved these changes Oct 28, 2025

View reviewed changes

akukanov reviewed Oct 28, 2025

View reviewed changes

Consistent naming of contraints and reserved_slots arguments

52a15d7

Signed-off-by: Isaev, Ilya <ilya.isaev@intel.com>

	The number of created ``task_arena`` is equal to the number of NUMA nodes available on the system.
	The number of created ``task_arena`` is equal to the number of NUMA nodes detected on the system.

		the ``task_arena`` objects. The ``numa_id`` value in ``constraints`` is automatically
		replaced with a separate value for each created arena.

-    the ``task_arena`` objects. The ``numa_id`` value in ``constraints`` is automatically
-    replaced with a separate value for each created arena.
+    the ``task_arena`` objects. The ``numa_id`` value specified in ``constraints`` is disregarded, and instead,
+    the corresponding ``numa_id`` from ``tbb::info::numa_nodes()`` is used during construction
+    of each particular ``task_arena``.

		std::vector<task_arena> create_numa_task_arenas(task_arena::constraints constraints = {},
		unsigned reserved_slots = 0);

Uh oh!

[oneTBB] Add a function to create a set of NUMA bound task arenas #650

Are you sure you want to change the base?

[oneTBB] Add a function to create a set of NUMA bound task arenas #650

Uh oh!

Conversation

isaevil commented Oct 2, 2025

Uh oh!

isaevil commented Oct 2, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akukanov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aleksei-fedotov left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

isaevil Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akukanov Oct 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

isaevil Oct 28, 2025 •

edited

Loading

akukanov Oct 28, 2025 •

edited

Loading