Skip to content

define graph.devices #854

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

define graph.devices #854

wants to merge 1 commit into from

Conversation

philloooo
Copy link
Contributor

@philloooo philloooo commented May 30, 2025 •
edited by pr-preview bot
Loading

fixes #836
hi, I've implemented this API on Chromium for both CoreML backend and DML backend and Tflite backend. So I am putting up the spec PR for discussion.

This re-introduces the MLDeviceType but in the context of a compiled graph. The use case shifts from letting users specify device preference to providing information to developers on where the graph would get run.

So I'd like to hear people's opinion on the naming for 'npu' under this context.

@fdwr @huningxin

Preview | Diff

@huningxin
Copy link
Contributor

Thanks @philloooo for Chromium prototyping and making this PR! It looks good to me.

DML backend,

As discussed in WebML WG Teleconference – 22 May 2025, Windows ML was announced as an evolution of DirectML. It would be good to know the implementability of Windows ML (ONNX Runtime).

@zolkis
Copy link
Collaborator

zolkis commented Jun 3, 2025

So I'd like to hear people's opinion on the naming for 'npu' under this context.

For the record, that has been strongly contested for the overloaded meaning of it in #815, which led to the proposal to use "gpu" or "gpu-like" labels.

However, I think we could experimentally use the name "npu" here -- when we note that it might mean many different HW implementations with wildly different capabilities. However, NPU is gaining more and more the meaning of being power efficient, low-throughput, moderately low-latency acceleration. We could diversify that naming later.

We should record (as developer use cases) what (programming) benefits could a client get from knowing that a node in a compiled graph is executing e.g. on NPU. As elaborated in this comment, it might reflect a past decision made on a temporary condition (i.e. there could be hidden parameters we don't record).

Anyway, I think this might be a good way to gather data/experience on how typical compiled graphs will look like, which will help improving the spec on the device selection (hints, for now, but later also capabilities ), so this proposal LGTM.

@philloooo philloooo mentioned this pull request Jun 3, 2025
Open
mwyrzykowski
mwyrzykowski suggested changes Jun 4, 2025
View reviewed changes
Copy link

@mwyrzykowski mwyrzykowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to see a use case as its not obvious why this is necessary to add to the API

@zolkis zolkis mentioned this pull request Jun 5, 2025
Open
zolkis pushed a commit to zolkis/webnn that referenced this pull request Jun 17, 2025
@google-labs-jules
Update device-selection-explainer.md
7736172 
This commit comprehensively updates the device selection explainer to reflect the latest discussions, API changes, and community feedback.

Key changes include:
- Updated Introduction and History sections to accurately reflect the removal of `MLDeviceType` from `MLContextOptions` (following PR webmachinelearning#809) and the shift towards hint-based, implementation-led device selection.
- Clarified the impact of key issues (webmachinelearning#749, webmachinelearning#302, webmachinelearning#350) and PRs (webmachinelearning#809, webmachinelearning#824, webmachinelearning#855) on the device selection strategy.
- Ensured the "Key use cases and requirements" section aligns with the current API, incorporating the device preference use cases from PR webmachinelearning#855.
- Updated JavaScript examples in "Scenarios, examples, design discussion" to be consistent with the current API, marking future/hypothetical features (like `opSupportLimitsPerDevice()` and a `fallback` option) with explanatory notes.
- Added new open questions based on recent discussions (e.g., issue webmachinelearning#836, PR webmachinelearning#854 regarding querying actual device usage).
- Refined the "Background thoughts" section, particularly the "Example Hardware Selection Guide," adding an editor's note about ongoing discussions (PR webmachinelearning#860).
- Corrected the "Considered alternatives" and "Minimum Viable Solution" sections to accurately represent the current and past approaches.
- Updated the "Next Phase Device Selection Solution" to clarify the status of proposals like `querySupport` (issue webmachinelearning#815) and the investigation of `graph.devices` (issue webmachinelearning#836, PR webmachinelearning#854).
- Performed a full proofread, correcting grammar, typos, and markdown formatting for improved clarity and consistency throughout the document.
zolkis pushed a commit to zolkis/webnn that referenced this pull request Jun 17, 2025
@google-labs-jules @zolkis
Update device-selection-explainer.md
989dd20 
This commit comprehensively updates the device selection explainer to reflect the latest discussions, API changes, and community feedback.

Key changes include:
- Updated Introduction and History sections to accurately reflect the removal of `MLDeviceType` from `MLContextOptions` (following PR webmachinelearning#809) and the shift towards hint-based, implementation-led device selection.
- Clarified the impact of key issues (webmachinelearning#749, webmachinelearning#302, webmachinelearning#350) and PRs (webmachinelearning#809, webmachinelearning#824, webmachinelearning#855) on the device selection strategy.
- Ensured the "Key use cases and requirements" section aligns with the current API, incorporating the device preference use cases from PR webmachinelearning#855.
- Updated JavaScript examples in "Scenarios, examples, design discussion" to be consistent with the current API, marking future/hypothetical features (like `opSupportLimitsPerDevice()` and a `fallback` option) with explanatory notes.
- Added new open questions based on recent discussions (e.g., issue webmachinelearning#836, PR webmachinelearning#854 regarding querying actual device usage).
- Refined the "Background thoughts" section, particularly the "Example Hardware Selection Guide," adding an editor's note about ongoing discussions (PR webmachinelearning#860).
- Corrected the "Considered alternatives" and "Minimum Viable Solution" sections to accurately represent the current and past approaches.
- Updated the "Next Phase Device Selection Solution" to clarify the status of proposals like `querySupport` (issue webmachinelearning#815) and the investigation of `graph.devices` (issue webmachinelearning#836, PR webmachinelearning#854).
- Performed a full proofread, correcting grammar, typos, and markdown formatting for improved clarity and consistency throughout the document.
zolkis pushed a commit to zolkis/webnn that referenced this pull request Jun 17, 2025
@google-labs-jules @zolkis
Update device-selection-explainer.md
8fecc6c 
This commit comprehensively updates the device selection explainer to reflect the latest discussions, API changes, and community feedback.

Key changes include:
- Updated Introduction and History sections to accurately reflect the removal of `MLDeviceType` from `MLContextOptions` (following PR webmachinelearning#809) and the shift towards hint-based, implementation-led device selection.
- Clarified the impact of key issues (webmachinelearning#749, webmachinelearning#302, webmachinelearning#350) and PRs (webmachinelearning#809, webmachinelearning#824, webmachinelearning#855) on the device selection strategy.
- Ensured the "Key use cases and requirements" section aligns with the current API, incorporating the device preference use cases from PR webmachinelearning#855.
- Updated JavaScript examples in "Scenarios, examples, design discussion" to be consistent with the current API, marking future/hypothetical features (like `opSupportLimitsPerDevice()` and a `fallback` option) with explanatory notes.
- Added new open questions based on recent discussions (e.g., issue webmachinelearning#836, PR webmachinelearning#854 regarding querying actual device usage).
- Refined the "Background thoughts" section, particularly the "Example Hardware Selection Guide," adding an editor's note about ongoing discussions (PR webmachinelearning#860).
- Corrected the "Considered alternatives" and "Minimum Viable Solution" sections to accurately represent the current and past approaches.
- Updated the "Next Phase Device Selection Solution" to clarify the status of proposals like `querySupport` (issue webmachinelearning#815) and the investigation of `graph.devices` (issue webmachinelearning#836, PR webmachinelearning#854).
- Performed a full proofread, correcting grammar, typos, and markdown formatting for improved clarity and consistency throughout the document.
anssiko pushed a commit that referenced this pull request Jun 27, 2025
@zolkis @google-labs-jules
Update with an example HW selection guide and new use cases (#860)
4927ec5 
This commit comprehensively updates the device selection explainer to reflect the latest discussions, API changes, and community feedback.

Key changes include:
- Updated Introduction and History sections to accurately reflect the removal of `MLDeviceType` from `MLContextOptions` (following PR #809) and the shift towards hint-based, implementation-led device selection.
- Clarified the impact of key issues (#749, #302, #350) and PRs (#809, #824, #855) on the device selection strategy.
- Ensured the "Key use cases and requirements" section aligns with the current API, incorporating the device preference use cases from PR #855.
- Updated JavaScript examples in "Scenarios, examples, design discussion" to be consistent with the current API, marking future/hypothetical features (like `opSupportLimitsPerDevice()` and a `fallback` option) with explanatory notes.
- Added new open questions based on recent discussions (e.g., issue #836, PR #854 regarding querying actual device usage).
- Refined the "Background thoughts" section, particularly the "Example Hardware Selection Guide," adding an editor's note about ongoing discussions (PR #860).
- Corrected the "Considered alternatives" and "Minimum Viable Solution" sections to accurately represent the current and past approaches.
- Updated the "Next Phase Device Selection Solution" to clarify the status of proposals like `querySupport` (issue #815) and the investigation of `graph.devices` (issue #836, PR #854).
- Performed a full proofread, correcting grammar, typos, and markdown formatting for improved clarity and consistency throughout the document.

Signed-off-by: Zoltan Kis <zoltan.kis@intel.com>
With thanks to Jules for proofreading:
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
anssiko pushed a commit that referenced this pull request Aug 8, 2025
@zolkis @google-labs-jules @anssiko
Update with an example HW selection guide and new use cases (#860)
540fb9b 
This commit comprehensively updates the device selection explainer to reflect the latest discussions, API changes, and community feedback.

Key changes include:
- Updated Introduction and History sections to accurately reflect the removal of `MLDeviceType` from `MLContextOptions` (following PR #809) and the shift towards hint-based, implementation-led device selection.
- Clarified the impact of key issues (#749, #302, #350) and PRs (#809, #824, #855) on the device selection strategy.
- Ensured the "Key use cases and requirements" section aligns with the current API, incorporating the device preference use cases from PR #855.
- Updated JavaScript examples in "Scenarios, examples, design discussion" to be consistent with the current API, marking future/hypothetical features (like `opSupportLimitsPerDevice()` and a `fallback` option) with explanatory notes.
- Added new open questions based on recent discussions (e.g., issue #836, PR #854 regarding querying actual device usage).
- Refined the "Background thoughts" section, particularly the "Example Hardware Selection Guide," adding an editor's note about ongoing discussions (PR #860).
- Corrected the "Considered alternatives" and "Minimum Viable Solution" sections to accurately represent the current and past approaches.
- Updated the "Next Phase Device Selection Solution" to clarify the status of proposals like `querySupport` (issue #815) and the investigation of `graph.devices` (issue #836, PR #854).
- Performed a full proofread, correcting grammar, typos, and markdown formatting for improved clarity and consistency throughout the document.

Signed-off-by: Zoltan Kis <zoltan.kis@intel.com>
With thanks to Jules for proofreading:
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@huningxin huningxin huningxin approved these changes

+1 more reviewer

@mwyrzykowski mwyrzykowski mwyrzykowski requested changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Get devices used for a graph after graph compilation
4 participants
@philloooo @huningxin @zolkis @mwyrzykowski