wasi_nn_tensorflowlite.cpp: make this compatible with wasmedge #4517
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
for wasi_ephemeral_nn,
* implement u8 input
* stop dealing with quantization.
* wasi-nn doesn't have a concept of quantization or pre/post-processing.
i can't think of any ways to make the backend perform zero-point/scale
processing without risking to break other applications.
* there seems to be applications which just use u8 inputs/outputs for
a quantized model. (see [1] for an example.)
for certain kinds of inputs/outputs, it usually just works.
this commit keeps the legacy wasi_nn logic intact for now.
tested with [1] with [2] applied.
WAMR with this patch:
```
Read graph weights, size in bytes: 3561598
[wasi_nn.c:297 WARNING] load_by_name_with_config() not found
[wasi_nn_tensorflowlite.cpp:272 WARNING] Default encoding is CPU.
Loaded graph into wasi-nn with ID: Graph#0
Read input tensor, size in bytes: 150528
1.) [166](198)Aix galericulata
2.) [34](1)Gallus gallus domesticus
3.) [158](1)Coccothraustes coccothraustes
4.) [778](1)Sitta europaea
5.) [819](1)Anas platyrhynchos
```
wasmedge:
```
Read graph weights, size in bytes: 3561598
Loaded graph into wasi-nn with ID: Graph#0
Read input tensor, size in bytes: 150528
1.) [166](198)Aix galericulata
2.) [34](1)Gallus gallus domesticus
3.) [158](1)Coccothraustes coccothraustes
4.) [778](1)Sitta europaea
5.) [819](1)Anas platyrhynchos
```
and "Aix galericulata" seems like a reasonable classification
of the image to my eyes.
[1] https://github.com/second-state/WasmEdge-WASINN-examples/tree/67f174bab59d98c1b52f7367ec0928701dc998f9/tflite-birds_v1-image
[2] second-state/WasmEdge-WASINN-examples#204
Related:
bytecodealliance#3555
bytecodealliance#2611
yamt
requested review from
loganek,
lum1n0us,
no1wudi,
TianlongLiang,
wenyongh and
xujuntwt95329
as code owners
yamt
added
wasi-nn
breaking-change
lum1n0us
approved these changes
Aug 1, 2025
TianlongLiang
approved these changes
Aug 1, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
for wasi_ephemeral_nn,
implement u8 input
stop dealing with quantization.
wasi-nn doesn't have a concept of quantization or pre/post-processing. i can't think of any ways to make the backend perform zero-point/scale processing without risking to break other applications.
there seems to be applications which just use u8 inputs/outputs for a quantized model. (see [1] for an example.) for certain kinds of inputs/outputs, it usually just works.
this commit keeps the legacy wasi_nn logic intact for now.
tested with [1] with [2] applied.
WAMR with this patch:
wasmedge:
and "Aix galericulata" seems like a reasonable classification of the image to my eyes.
[1] https://github.com/second-state/WasmEdge-WASINN-examples/tree/67f174bab59d98c1b52f7367ec0928701dc998f9/tflite-birds_v1-image
[2] second-state/WasmEdge-WASINN-examples#204
Related:
#3555 #2611