How does gemini nano work on qcom Npu/Gpu

[AndreaChiChengdu]

I have a question. From the documentation and introduction, ai edge sdk api is used to access AICORE, while gemini nano is in AICORE. How does AICORE connect to Qualcomm npu hardware? Is it through runtime in AICORE or liteRT gpu delegate（tflite runtime） or Qualcomm's own qnn htp/gpu delegate? It's not mentioned in the document or the video.
thanks

[AndreaChiChengdu]

@zhouyiself Is AICORE based on liteRT to connect to the Qualcomm acceleration library or is there another implementation in AICORE? Thank you very much
#7 (comment)

[Flyfish233]

AICore on Galaxy device:

lib/arm64-v8a/libQnnHtp.so
lib/arm64-v8a/libQnnHtpV73Skel.so
lib/arm64-v8a/libQnnHtpV73Stub.so
lib/arm64-v8a/libQnnHtpV75Skel.so
lib/arm64-v8a/libQnnHtpV75Stub.so
lib/arm64-v8a/libQnnSystem.so
lib/arm64-v8a/libllm_qnn_jni.so

AICore thirdpartyeap:

lib/arm64-v8a/libllm_edgetpu_jni.so
lib/arm64-v8a/libllm_ggml_jni.so
lib/arm64-v8a/libulm1b_ggml_jni.so

That's enough to answer this issue.

[AndreaChiChengdu]

AICore on Galaxy device:

lib/arm64-v8a/libQnnHtp.so
lib/arm64-v8a/libQnnHtpV73Skel.so
lib/arm64-v8a/libQnnHtpV73Stub.so
lib/arm64-v8a/libQnnHtpV75Skel.so
lib/arm64-v8a/libQnnHtpV75Stub.so
lib/arm64-v8a/libQnnSystem.so
lib/arm64-v8a/libllm_qnn_jni.so

AICore thirdpartyeap:

lib/arm64-v8a/libllm_edgetpu_jni.so
lib/arm64-v8a/libllm_ggml_jni.so
lib/arm64-v8a/libulm1b_ggml_jni.so

That's enough to answer this issue.

Thanks a lot, but it looks like the pixel and sm8650 have different software paths? pixel uses ggml, Qualcomm npu as far as I know there are only two ways of tflite qnnhtp delegate and qnn offline compilation, and all signs show that aicore does not seem to use tflite runtime, which is where I do not understand. Thank you for your reply

[Flyfish233]

AICore has several versions depending on the SoC, on Galaxy devices they may wrapped Qualcomm Genie SDK because that's the fastest API on a modern Qualcomm device with NPU, and they might not care about compabilities, so they probably won't use TFlite or GPUs.

[calren]

AICore does not use TFLite / LiteRT. Is there a specific use case you're trying to enable?