Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have limit on the line width? Proper alignment will help avoiding unnecessary horizontal scrollings and make the rendering neater.

Consider:

Image Image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HTML page looks like this:
image
Seems acceptable, and I am not sure if manual line break can do better in this case.

Copy link
Contributor Author

@isaevil isaevil Oct 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know, you can't make indentation when using .. cpp::function with line breaks, so at best it will end up like this:

std::vector<task_arena> create_numa_task_arenas(
task_arena::constraints constraints = {}, 
int reserved_slots = 0)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can reduce the clutter by using shorter names. For example, I find shortening of constraints function parameter to just c attractive, because:

  1. The task_arena::constraints type conveys the semantics.
  2. The description from where the parameter is referred to mentions that these are constraints.
  3. The description of the function is short enough and scoped, so c could act similarly to constructs like loop counters. Meaning, that readers are unlikely to forget what that c is about.

Similarly, the a_priority can be reduced to just p without loosing much of the readability.

This should not be overused though. For example, I would not contract the reserved_slots because its type does not convey the meaning, so the name is essential here.

Original file line number Diff line number Diff line change
Expand Up @@ -42,18 +42,18 @@ A class that represents an explicit, user-managed task scheduler arena.
int max_threads_per_core = task_arena::automatic;
};

task_arena(int max_concurrency = automatic, unsigned reserved_for_masters = 1,
priority a_priority = priority::normal);
task_arena(constraints a_constraints, unsigned reserved_for_masters = 1,
priority a_priority = priority::normal);
task_arena(int max_concurrency = automatic, unsigned reserved_slots = 1,
priority a_priority = priority::normal);
task_arena(constraints constraints_, unsigned reserved_slots = 1,
priority a_priority = priority::normal);
task_arena(const task_arena &s);
explicit task_arena(oneapi::tbb::attach);
~task_arena();

void initialize();
void initialize(int max_concurrency, unsigned reserved_for_masters = 1,
void initialize(int max_concurrency, unsigned reserved_slots = 1,
priority a_priority = priority::normal);
void initialize(constraints a_constraints, unsigned reserved_for_masters = 1,
void initialize(constraints constraints_, unsigned reserved_slots = 1,
priority a_priority = priority::normal);
void initialize(oneapi::tbb::attach);

Expand All @@ -71,6 +71,9 @@ A class that represents an explicit, user-managed task scheduler arena.
task_group_status task_arena::wait_for(task_group& tg);
};

std::vector<task_arena> create_numa_task_arenas(task_arena::constraints constraints_ = {},
unsigned reserved_slots = 0);

} // namespace tbb
} // namespace oneapi

Expand Down Expand Up @@ -182,28 +185,28 @@ Member types and constants
Member functions
----------------

.. cpp:function:: task_arena(int max_concurrency = automatic, unsigned reserved_for_masters = 1, priority a_priority = priority::normal)
.. cpp:function:: task_arena(int max_concurrency = automatic, unsigned reserved_slots = 1, priority a_priority = priority::normal)

Creates a ``task_arena`` with a certain concurrency limit (``max_concurrency``) and priority
(``a_priority``). Some portion of the limit can be reserved for application threads with
``reserved_for_masters``. The amount for reservation cannot exceed the limit.
``reserved_slots``. The amount for reservation cannot exceed the limit.

.. caution::

If ``max_concurrency`` and ``reserved_for_masters`` are
If ``max_concurrency`` and ``reserved_slots`` are
explicitly set to be equal and greater than 1, oneTBB worker threads will never
join the arena. As a result, the execution guarantee for enqueued tasks is not valid
in such arena. Do not use ``task_arena::enqueue()`` with an arena set to have no worker threads.

.. cpp:function:: task_arena(constraints a_constraints, unsigned reserved_for_masters = 1, priority a_priority = priority::normal)
.. cpp:function:: task_arena(constraints constraints_, unsigned reserved_slots = 1, priority a_priority = priority::normal)

Creates a ``task_arena`` with a certain constraints(``a_constraints``) and priority
Creates a ``task_arena`` with a certain constraints(``constraints_``) and priority
(``a_priority``). Some portion of the limit can be reserved for application threads with
``reserved_for_masters``. The amount for reservation cannot exceed the concurrency limit specified in ``constraints``.
``reserved_slots``. The amount for reservation cannot exceed the concurrency limit specified in ``constraints``.

.. caution::

If ``constraints::max_concurrency`` and ``reserved_for_masters`` are
If ``constraints::max_concurrency`` and ``reserved_slots`` are
explicitly set to be equal and greater than 1, oneTBB worker threads will never
join the arena. As a result, the execution guarantee for enqueued tasks is not valid
in such arena. Do not use ``task_arena::enqueue()`` with an arena set to have no worker threads.
Expand Down Expand Up @@ -239,11 +242,11 @@ Member functions

After the call to ``initialize``, the arena parameters are fixed and cannot be changed.

.. cpp:function:: void initialize(int max_concurrency, unsigned reserved_for_masters = 1, priority a_priority = priority::normal)
.. cpp:function:: void initialize(int max_concurrency, unsigned reserved_slots = 1, priority a_priority = priority::normal)

Same as above, but overrides previous arena parameters.

.. cpp:function:: void initialize(constraints a_constraints, unsigned reserved_for_masters = 1, priority a_priority = priority::normal)
.. cpp:function:: void initialize(constraints constraints_, unsigned reserved_slots = 1, priority a_priority = priority::normal)

Same as above.

Expand Down Expand Up @@ -329,6 +332,26 @@ Member functions

The behavior of this function is equivalent to ``this->execute([&tg]{ return tg.wait(); })``.

Non-member Functions
--------------------

.. cpp:function:: std::vector<task_arena> create_numa_task_arenas(task_arena::constraints constraints_ = {}, unsigned reserved_slots = 0)

Returns a ``std::vector`` of non-initialized ``task_arena`` objects, each bound to a separate NUMA node.
The number of created ``task_arena`` instances is equal to the number of NUMA nodes on the system,
as determined by ``tbb::info::numa_nodes()``.

If an error occurs during system information discovery,
returns a ``std::vector`` containing a single ``task_arena`` object created as
``task_arena(constraints_.set_numa_id(task_arena::automatic), reserved_slots)``.

The ``constraints_`` argument can be specified to apply additional limitations to threads in
the ``task_arena`` objects. For each created arena, the ``numa_id`` value in ``constraints_``
is automatically set to the corresponding NUMA node ID from ``tbb::info::numa_nodes()``.

The ``reserved_slots`` argument allows reserving a specified number of slots in
each ``task_arena`` object for application threads. By default, no slots are reserved.

Example
-------

Expand All @@ -343,22 +366,21 @@ to the corresponding NUMA node.
#include <vector>

int main() {
std::vector<oneapi::tbb::numa_node_id> numa_nodes = oneapi::tbb::info::numa_nodes();
std::vector<oneapi::tbb::task_arena> arenas(numa_nodes.size());
std::vector<oneapi::tbb::task_group> task_groups(numa_nodes.size());

for (int i = 0; i < numa_nodes.size(); i++) {
arenas[i].initialize(oneapi::tbb::task_arena::constraints(numa_nodes[i]));
}
std::vector<oneapi::tbb::task_arena> arenas = oneapi::tbb::create_numa_task_arenas();
std::vector<oneapi::tbb::task_group> task_groups(arenas.size()-1);

for (int i = 0; i < numa_nodes.size(); i++) {
for (int i = 1; i < arenas.size(); i++) {
arenas[i].enqueue([]{
/* executed by a thread pinned to the specified NUMA node */
}, task_groups[i]);
}, task_groups[i-1]);
}

for (int i = 0; i < numa_nodes.size(); i++) {
arenas[i].wait_for(task_groups[i]);
arenas[0].execute([] {
/* executed by the main thread pinned to the NUMA node for arenas[0] */
});

for (int i = 1; i < arenas.size(); i++) {
arenas[i].wait_for(task_groups[i-1]);
}

return 0;
Expand Down
Loading