Problem loading custom model

[Corallo]

Hi, I have the same issue when I loaded my own tflite model.
I have already replaced the path to my model in the Dockerfile.model and env.aarch64.artpec8:

Dockerfile.model:

# Get custom model
ADD ./models/yolov4/train-608_best.tflite models/
ADD ./models/yolov4/labels.txt models/

env.aarch64.artpec8:

MODEL_PATH=/models/train-608_best.tflite
OBJECT_LIST_PATH=/models/labels.txt
INFERENCE_SERVER_IMAGE=axisecp/acap-runtime:aarch64-containerized
INFERENCE_SERVER_ARGS="-m /models/train-608_best.tflite -j 12" 

There is the error I encountered:

error message:

ERROR in Inference: Failed to load model train-608_best.tflite (Could not load model: Asynchronous connection has been closed)

<_InactiveRpcError of RPC that terminated with:
object-detector-python_1 | status = StatusCode.UNAVAILABLE
object-detector-python_1 | details = "failed to connect to all addresses"
object-detector-python_1 | debug_error_string = "{"created":"@1659425957.027178280","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3217,"referenced_errors":[{"created":"@1659425957.027173640","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":165,"grpc_status":14}]}"

Environment

• Axis device model: P3265-LVE
• Axis device firmware version: 10.11.76 (also tried 10.10.73 but didn't work)
• SDK VERSION: 1.2.1
• docker daemon with Compose: 1.2.3

Please help me, thanks in advance.

Originally posted by @JENNSHIUAN in #50 (comment)

[Corallo]

Hi @JENNSHIUAN
I converted your question to a new discussion, as it is not related to the problem in that issue.
In your case, It seems that the problem is that the model that you have can't be loaded on the camera.
If you are using yolov4 it is likely that you have compatibility issues, as it is not a model made to run on edge devices.
You could check by SSH into the camera and running journalctl -u larod

[Corallo]

Hi @JENNSHIUAN

It seems that the model you have is quantized with dynamic range quantization, while it should instead be quantized with integer quantization. See: https://www.tensorflow.org/lite/performance/post_training_quant
Did you try the int8 version of the quantization from that repository?

[JENNSHIUAN]

Sure, I also tried to quantize to uint8 using tensorflow API tflite converter with the following tutorial: Tensorflowlite-Samples, it can not work, though. I'm frustrated about that. Is there any example model to run yolo on axis camera or should I do Quantization aware training with it?

[Corallo]

Can you share the int8 version of the converted model?
If you run journalctl -u larod when loading that model, do you have the same error in the log?

Unfortunately no, Yolo is not a network optimized for edge devices, and we never made an example with it, we should however at least have a proper error log that show to users what is the problem with the model instead of just crashing.

[JENNSHIUAN]

I've already sent an updated one to you, please check.
The error message is shown:

ERROR in Inference: Failed to load model train-608_best_uint8_tflite_convert_fromh5.tflite (Could not load model: Could not verify and build model from buffer)

BTW, I'm trying to retrain a new model for ssd mobilenetv2. As @igor-rud-kr mentioned, in order to avoid the same issue, is there a tutorial on how to re-train SSD MobileNet V2 detector?

[Corallo]

We have a general guide about how to retrain models to keep compatible with artpec8 camera, it is not ssd mobilenet specific.
We recommend you to use the Tensorflow Object Detector API they have a prebuild ssd mobilenetv2 model
that you can retrain. If you care about the performance in terms of inference time of your model, take a look at the guide, because artpec8 cameras perform better if you quantize the model properly

[Corallo]

@JENNSHIUAN it took a long time for us to finish the investigation, in the end unfortunately we do not support this version of yolo.
The quantization is not done properly.
#65

[Corallo]

Hi again,
It is now possible to run Yolov5 on Artpec-8 on devices running Axis OS >= 11.7
See the guide here