diff --git a/bin/pagefind.bak b/bin/pagefind.bak new file mode 100755 index 0000000000..38b593b376 Binary files /dev/null and b/bin/pagefind.bak differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/1-overview.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/1-overview.md new file mode 100644 index 0000000000..788cb474a7 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/1-overview.md @@ -0,0 +1,55 @@ +--- +title: Overview +weight: 2 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Hardware Overview - NXP's FRDM i.MX 93 Board + +Selecting the best hardware for machine learning (ML) models depends on effective tools. You can visualize ML performance early in the development cycle by using NXP's [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) board. + +
+ + +*Unboxing NXP's FRDM i.MX 93 board* +
+ +![NXP FRDM i.MX 93 Board SoC Highlighted alt-text#center](./nxp-frdm-imx-93-board-soc-highlighted.png "Arm Ethos-U65 NPU location") + +### NXP's FRDM i.MX 93 Processor Decoded + +![i.MX 93 Processor SoC alt-text#center](./imx-93-application-processor-soc.png "NXP's FRDM i.MX 93 processor") + +**NXP's Processor Labeling Convention:** +|Line|Meaning| +|----|-------| +|MIMX9352|• MI – Microcontroller IC
• MX93 – i.MX 93 family
• 52 – Variant:
• Dual-core Arm Cortex-A55
• Single Cortex-M33
• Includes **Ethos-U65 NPU**| +|CVVXMAB|• C - Commercial temperature grade (0°C to 95°C)
• VVX - Indicates package type and pinout (BGA, pitch, etc.)
• MAB - Specific configuration (e.g., NPU present, security level, memory interfaces) +| +|1P87F|• Silicon mask set identifier| +|SBBM2410E|• NXP traceability code| + +## Software Overview - NXP's MCUXpresso IDE + +NXP generously provides free software for working with their boards, the [MCUXpresso Integrated Development Environment (IDE)](https://www.nxp.com/design/design-center/software/development-software/mcuxpresso-software-and-tools-/mcuxpresso-integrated-development-environment-ide:MCUXpresso-IDE). In this learning path, you will instead use [MCUXpresso for Visual Studio Code](https://www.nxp.com/design/design-center/software/development-software/mcuxpresso-software-and-tools-/mcuxpresso-for-visual-studio-code:MCUXPRESSO-VSC). + +## Software Overview - Visual Studio Code + +[Visual Studio Code](https://code.visualstudio.com/) is a free integrated development environment provided by Microsoft. It is platform independent, full featured, and accomodating of many engineering frameworks. You will use Visual Studio Code to both configure NXP's software and connect to NXP's hardware. + +## Software Overview - TinyML. + +This Learning Path uses TinyML. TinyML is machine learning tailored to function on devices with limited resources, constrained memory, low power, and fewer processing capabilities. + +For a learning path focused on creating and deploying your own TinyML models, please see [Introduction to TinyML on Arm using PyTorch and ExecuTorch](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/) + +## Benefits and applications + +NPUs, like Arm's [Ethos-U65](https://www.arm.com/products/silicon-ip-cpu/ethos/ethos-u65) NPU are available on physical devices specifically made for developers. Development boards like NXP's [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) also connect to displays via a HDMI cable. Additionally the board accepts video inputs. This is useful for for ML performance visualization due to: +- visual confirmation that your ML model is running on the physical device +- image and video inputs for computer vision models running on the device +- clearly indicated instruction counts +- confirmation of total execution time and +- visually appealing output for prototypes and demos diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/10-deploy-executorchrunner-nxp-board.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/10-deploy-executorchrunner-nxp-board.md new file mode 100644 index 0000000000..ccc9e90bfa --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/10-deploy-executorchrunner-nxp-board.md @@ -0,0 +1,288 @@ +--- +title: Deploy and test on FRDM-IMX93 +weight: 11 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Connect to the FRDM-IMX93 board + +The FRDM-IMX93 board runs Linux on the Cortex-A55 cores. You need network or serial access to deploy the firmware. + +Find your board's IP address using the serial console or check your router's DHCP leases. + +Connect via SSH: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +```bash +ssh root@192.168.1.24 +``` + +Alternative with PuTTY on Windows: +- Host: `192.168.1.24` +- Port: `22` +- Connection type: SSH +- Username: `root` +{{< /tab >}} +{{< tab header="macOS" >}} +```bash +ssh root@192.168.1.24 +``` +{{< /tab >}} +{{< /tabpane >}} + +Replace `192.168.1.24` with your board's IP address. + +## Copy the firmware to the board + +Copy the built firmware file to the board's firmware directory: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +```bash +scp debug/executorch_runner_cm33.elf root@192.168.1.24:/lib/firmware/ +``` +{{< /tab >}} +{{< tab header="macOS" >}} +```bash +scp debug/executorch_runner_cm33.elf root@192.168.1.24:/lib/firmware/ +``` +{{< /tab >}} +{{< /tabpane >}} + +Verify the file was copied: + +```bash { command_line="root@frdm-imx93" output_lines="2" } +ls -lh /lib/firmware/executorch_runner_cm33.elf +-rw-r--r-- 1 root root 601K Oct 24 10:30 /lib/firmware/executorch_runner_cm33.elf +``` + +## Load the firmware on Cortex-M33 + +The Cortex-M33 firmware is managed by the RemoteProc framework running on Linux. + +Stop any currently running firmware: + +```bash { command_line="root@frdm-imx93" } +echo stop > /sys/class/remoteproc/remoteproc0/state +``` + +Set the new firmware: + +```bash { command_line="root@frdm-imx93" } +echo executorch_runner_cm33.elf > /sys/class/remoteproc/remoteproc0/firmware +``` + +Start the Cortex-M33 with the new firmware: + +```bash { command_line="root@frdm-imx93" } +echo start > /sys/class/remoteproc/remoteproc0/state +``` + +Verify the firmware loaded successfully: + +```bash { command_line="root@frdm-imx93" output_lines="2-5" } +dmesg | grep remoteproc | tail -n 5 +[12345.678] remoteproc remoteproc0: powering up imx-rproc +[12345.679] remoteproc remoteproc0: Booting fw image executorch_runner_cm33.elf, size 614984 +[12345.680] remoteproc remoteproc0: header-less resource table +[12345.681] remoteproc remoteproc0: remote processor imx-rproc is now up +``` + +The message "remote processor imx-rproc is now up" confirms successful loading. + +## Load a model to DDR memory + +The executor_runner loads `.pte` model files from DDR memory at address 0x80100000. + +Copy your `.pte` model to the board: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +```bash +scp model.pte root@192.168.1.24:/tmp/ +``` +{{< /tab >}} +{{< tab header="macOS" >}} +```bash +scp model.pte root@192.168.1.24:/tmp/ +``` +{{< /tab >}} +{{< /tabpane >}} + +Write the model to DDR memory: + +```bash { command_line="root@frdm-imx93" } +dd if=/tmp/model.pte of=/dev/mem bs=1M seek=2049 +``` + +The seek value of 2049 corresponds to address 0x80100000 (2049 MB = 0x801 in hex). + +Verify the model was written: + +```bash { command_line="root@frdm-imx93" output_lines="2-5" } +xxd -l 64 -s 0x80100000 /dev/mem +80100000: 504b 0304 1400 0000 0800 0000 2100 a3b4 PK..........!... +80100010: 7d92 5801 0000 6c04 0000 1400 0000 7661 }.X...l.......va +80100020: 6c75 652f 7061 7261 6d73 2e70 6b6c 6500 lue/params.pkl. +80100030: ed52 cd4b 0241 1cfd 66de 49b6 9369 1ad9 .R.K.A..f.I..i.. +``` + +Non-zero bytes confirm the model is present in memory. + +## Monitor Cortex-M33 output + +The executor_runner outputs debug information via UART. Connect a USB-to-serial adapter to the M33 UART pins on the FRDM board. + +Open a serial terminal (115200 baud, 8N1): + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +```bash +screen /dev/ttyUSB0 115200 +``` + +Alternative with minicom: +```bash +minicom -D /dev/ttyUSB0 -b 115200 +``` +{{< /tab >}} +{{< tab header="macOS" >}} +```bash +screen /dev/tty.usbserial-* 115200 +``` + +Alternative with minicom: +```bash +minicom -D /dev/tty.usbserial-* -b 115200 +``` +{{< /tab >}} +{{< /tabpane >}} + +You should see output from the ExecuTorch runtime: + +```output +ExecuTorch Runtime Starting... +Loading model from 0x80100000 +Model loaded successfully +Initializing Ethos-U NPU delegate +NPU initialized +Running inference... +Inference complete: 45.2ms +``` + +{{% notice Tip %}} +If you don't see UART output, verify the serial connection settings (115200 baud, 8N1) and check that the UART pins are correctly connected. +{{% /notice %}} + +## Test inference + +The executor_runner automatically runs inference when it starts. Check the UART output for inference results and timing. + +To restart inference, you can reload the firmware: + +```bash { command_line="root@frdm-imx93" } +echo stop > /sys/class/remoteproc/remoteproc0/state +echo start > /sys/class/remoteproc/remoteproc0/state +``` + +Monitor the UART console to see the new inference run. + +## Verify deployment success + +Confirm your deployment is working correctly: + +1. **RemoteProc status shows "running":** + +```bash { command_line="root@frdm-imx93" output_lines="2" } +cat /sys/class/remoteproc/remoteproc0/state +running +``` + +2. **Firmware is loaded:** + +```bash { command_line="root@frdm-imx93" output_lines="2" } +cat /sys/class/remoteproc/remoteproc0/firmware +executorch_runner_cm33.elf +``` + +3. **Model is in DDR memory** (non-zero bytes at 0x80100000) + +4. **UART shows inference output** with timing information + +## Troubleshooting + +**RemoteProc fails to load firmware:** + +Check file permissions: + +```bash { command_line="root@frdm-imx93" } +chmod 644 /lib/firmware/executorch_runner_cm33.elf +``` + +Verify the file exists: + +```bash { command_line="root@frdm-imx93" } +ls -la /lib/firmware/executorch_runner_cm33.elf +``` + +**Model not found error:** + +Verify the model was written to memory: + +```bash { command_line="root@frdm-imx93" } +xxd -l 256 -s 0x80100000 /dev/mem | head +``` + +If all zeros, re-run the `dd` command to write the model. + +**No UART output:** + +Check the serial connection: +- Baud rate: 115200 +- Data bits: 8 +- Parity: None +- Stop bits: 1 + +Try a different USB port or serial terminal program. + +**Firmware crashes or hangs:** + +Check kernel logs for errors: + +```bash { command_line="root@frdm-imx93" } +dmesg | grep -i error | tail +``` + +This might indicate memory configuration issues. Reduce the memory pool sizes in `CMakeLists.txt` and rebuild. + +## Update the firmware + +To deploy a new version of the firmware: + +1. Build the updated firmware on your development machine +2. Copy to the board: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +```bash +scp debug/executorch_runner_cm33.elf root@:/lib/firmware/ +``` +{{< /tab >}} +{{< tab header="macOS" >}} +```bash +scp debug/executorch_runner_cm33.elf root@:/lib/firmware/ +``` +{{< /tab >}} +{{< /tabpane >}} + +3. Restart RemoteProc: + +```bash { command_line="root@frdm-imx93" } +echo stop > /sys/class/remoteproc/remoteproc0/state +echo start > /sys/class/remoteproc/remoteproc0/state +``` + +4. Monitor UART output to verify the new firmware is running diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/2-boot-nxp.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/2-boot-nxp.md new file mode 100644 index 0000000000..831aa0ec48 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/2-boot-nxp.md @@ -0,0 +1,81 @@ +--- +# User change +title: "Boot the NXP FRDM i.MX 93 Board" + +weight: 3 + +# Do not modify these elements +layout: "learningpathall" +--- + +In this section, you will prepare the NXP [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) board for ML development. + +## Unbox the NXP Board + +Follow NXP's getting started instructions: [Getting Started with FRDM-IMX93](https://www.nxp.com/document/guide/getting-started-with-frdm-imx93:GS-FRDM-IMX93): +* Stop when you complete section "1.6 Connect Power Supply" + +## Connect to the NXP Board + +Prior to logging in to the NXP board, you need to configure `picocom`. This allows you to connect to the board using a USB cable. + +{{% notice macOS %}} + +1. Install the Silicon Labs driver: + + https://www.silabs.com/developer-tools/usb-to-uart-bridge-vcp-drivers?tab=downloads + +2. Install [picocom](https://github.com/npat-efault/picocom): + ```bash + brew install picocom + ``` +{{% /notice %}} + +1. Establish a USB-to-UART (serial) connection: + - Connect the board's "DEBUG" USB-C connector to your laptop + - Find the NXP board's USB connections in your computer's terminal: + ```bash { output_lines = "2-7" } + ls /dev/tty.* + # output lines + ... + /dev/tty.debug-console + /dev/tty.usbmodem56D70442811 + /dev/tty.usbmodem56D70442813 + ... + ``` + + - Connect to the NXP board: + ```bash { output_lines = "2-5" } + sudo picocom -b 115200 /dev/tty.usbmodem56D70442811 + # output lines + picocom v3.1 + ... + Terminal ready + ``` +2. Log in to the NXP board: + - Connect the board's "POWER" USB-C connector to your laptop + - At this point you should see one red and one white light on the board + - Next you should see scrolling text in your `picocom` window, as the NXP board boots + - The last line should say `login:` + ```bash { output_lines = "1-9" } + # output lines + ... + [ OK ] Reached target Graphical Interface. + Starting Record Runlevel Change in UTMP... + [ OK ] Finished Record Runlevel Change in UTMP. + + NXP i.MX Release Distro 6.6-scarthgap imx93frdm ttyLP0 + + imx93frdm login: + ``` + +3. [Optional] Troubleshooting: + - Restart the NXP board, to get to the `login:` prompt: + - Hold the NXP board's power button for 2-seconds, until the lights turn off + - Hold the NXP board's power button again for 2-seconds, until the lights turn on + +## [Optional] Run the Built-In NXP Demos +* Connect the NXP board to a monitor via HDMI +* Connect a mouse to the NXP board's USB-A port + +![NXP board built-in ML demos alt-text#center](./nxp-board-built-in-ml-demos.png "NXP board built-in ML demos") diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/4-environment-setup.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/4-environment-setup.md new file mode 100644 index 0000000000..47aa4bc6d2 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/4-environment-setup.md @@ -0,0 +1,120 @@ +--- +# User change +title: "Enviroment Setup" + +weight: 5 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + +For detailed instructions on setting up your ExecuTorch build environment, please see the official PyTorch documentation: [Environment Setup](https://docs.pytorch.org/executorch/stable/using-executorch-building-from-source.html#environment-setup) + +{{% notice macOS %}} + +Use a Docker container to build ExecuTorch: +* The [Arm GNU Toolchain](https://developer.arm.com/Tools%20and%20Software/GNU%20Toolchain) currently does not have a "AArch64 GNU/Linux target" for macOS +* You will use this toolchain's `gcc-aarch64-linux-gnu` and `g++-aarch64-linux-gnu` compilers on the next page of this learning path + +1. Install and start [Docker Desktop](https://www.docker.com/) + +2. Create a directory for building a `ubuntu-24-container`: + + ```bash + mkdir ubuntu-24-container + ``` + +3. Create a `dockerfile` in the `ubuntu-24-container` directory: + + ```bash + cd ubuntu-24-container + touch Dockerfile + ``` + +4. Add the following commands to your `Dockerfile`: + + ```dockerfile + FROM ubuntu:24.04 + + ENV DEBIAN_FRONTEND=noninteractive + + RUN apt update -y && \ + apt install -y \ + software-properties-common \ + curl vim git + ``` + + The `ubuntu:24.04` container image includes Python 3.12, which will be used for this learning path. + +5. Create the `ubuntu-24-container`: + + ```bash + docker build -t ubuntu-24-container . + ``` + +6. Run the `ubuntu-24-container`: + + ```bash { output_lines = "2-3" } + docker run -it ubuntu-24-container /bin/bash + # Output will be the Docker container prompt + ubuntu@:/# + ``` + + [OPTIONAL] If you already have an existing container: + - Get the existing CONTAINER ID: + ```bash { output_lines = "2-4" } + docker ps -a + # Output + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + 0123456789ab ubuntu-24-container "/bin/bash" 27 hours ago Exited (255) 59 minutes ago. container_name + ``` + - Log in to the existing container: + ```bash + docker start 0123456789ab + docker exec -it 0123456789ab /bin/bash + ``` + +{{% /notice %}} + +After logging in to the Docker container, navigate to the ubuntu home directory: + +```bash +cd /home/ubuntu +``` + +1. **Install dependencies:** + + ```bash { output_lines = "1" } + # Use "sudo apt ..." if you are not logged in as root + apt update + apt install -y \ + python-is-python3 python3.12-dev python3.12-venv \ + gcc g++ \ + make cmake \ + build-essential \ + ninja-build \ + libboost-all-dev + ``` + +2. Clone ExecuTorch: + ```bash + git clone https://github.com/pytorch/executorch.git + cd executorch + git fetch --tags + git checkout v1.0.0 + git submodule sync + git submodule update --init --recursive + ``` + +3. Create a Virtual Environment: + ```bash { output_lines = "3" } + python3 -m venv .venv + source .venv/bin/activate + # Your prompt will prefix with (.venv) + ``` + +4. Configure your git username and email globally: + ```bash + git config --global user.email "you@example.com" + git config --global user.name "Your Name" + ``` diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/6-build-executorch.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/6-build-executorch.md new file mode 100644 index 0000000000..f8dbf8b6a0 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/6-build-executorch.md @@ -0,0 +1,59 @@ +--- +# User change +title: "Build ExecuTorch" + +weight: 7 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + +For a full tutorial on building ExecuTorch please see learning path [Introduction to TinyML on Arm using PyTorch and ExecuTorch](/learning-paths/embedded-and-microcontrollers/introduction-to-tinyml-on-arm/) + +1. Build and install the `executorch` pip package from Source: + + ```bash + git submodule sync + git submodule update --init --recursive + ./install_executorch.sh + ``` + +# Troubleshooting + +1. Allocate at least 4 GB of swap space: + ```bash + fallocate -l 4G /swapfile + chmod 600 /swapfile + mkswap /swapfile + swapon /swapfile + ``` + [optional] Deallocate the swap space after you complete this learning path: + ```bash + swapoff /swapfile + rm /swapfile + ``` + + {{% notice macOS %}} + + Increase the "Swap" space in Docker settings to 4 GB: + ![Increase the swap space in Docker settings to 4 GB alt-text#center](./increase-swap-space-to-4-gb.jpg "Increase the swap space in Docker settings to 4 GB") + + {{% /notice %}} + +2. Kill the `buck2` process: + ```bash + ps aux | grep buck + pkill -f buck + ``` + +3. Clean the build environment and reinitialize all submodules: + ```bash + ./install_executorch.sh --clean + git submodule sync + git submodule update --init --recursive + ``` + +4. Try `install_executorch.sh` again, in development mode: + ```bash + ./install_executorch.sh + ``` \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/7-build-executorch-pte.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/7-build-executorch-pte.md new file mode 100644 index 0000000000..4bec2ad216 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/7-build-executorch-pte.md @@ -0,0 +1,201 @@ +--- +# User change +title: "Build the ExecuTorch .pte " + +weight: 8 # 1 is first, 2 is second, etc. + +# Do not modify these elements +layout: "learningpathall" +--- + +Embedded systems like the NXP board require two ExecuTorch runtime components: a `.pte` file and an `exeuctor_runner` file. + +**ExecuTorch Runtime Files for Embedded Systems** +|Component|Role in Deployment|What It Contains|Why It’s Required| +|---------|------------------|----------------|-----------------| +|**`.pte file`** (e.g., `mv2_arm_delegate_ethos-u55-256.pte`)|The model itself, exported from ExecuTorch|Serialized and quantized operator graph + weights + metadata|Provides the neural network to be executed| +|**`executor_runner`** (binary [ELF](https://www.netbsd.org/docs/elf.html) file)|The runtime program that runs the .pte file|C++ application that loads the .pte, prepares buffers, and calls the NPU or CPU backend|Provides the execution engine and hardware access logic| + + +
+
+
+┌───────────────────────────────────────────────────┐
+│                                                   │
+│                Host Development                   │
+│         (e.g., Linux or macOS+Docker)             │
+│                                                   │
+│  [Model export / compilation with ExecuTorch]     │
+│                                                   │
+│     ┌───────────────────┐        ┌───────────┐    │
+│     │                   │        │           │    │
+│     │  executor_runner  │        │  .pte     │    │
+│     │  (ELF binary)     │        │ (model)   │    │
+│     │                   │        │           │    │
+│     └───────────┬───────┘        └─────┬─────┘    │
+│                 │                      │          │
+└─────────────────┼──────────────────────┼──────────┘
+       │ SCP/serial transfer  │
+       │                      │
+       ▼                      ▼
+┌───────────────────────────────────────────────────┐
+│                                                   │
+│            NXP i.MX93 Embedded Board              │
+│                                                   │
+│                                                   │
+│  ┌───────────────────────────────────────────┐    │
+│  │   executor_runner (runtime binary)        │    │
+│  │                                           │    │
+│  │    ┌───────────────────────────────┐      │    │
+│  │    │ Load .pte (model)             │      │    │
+│  │    └───────────────┬───────────────┘      │    │
+│  │                    │                      │    │
+│  │                    ▼                      │    │
+│  │    ┌───────────────────────────────┐      │    │
+│  │    │ Initialize hardware (CPU/NPU) │      │    │
+│  │    └───────────────┬───────────────┘      │    │
+│  │                    │                      │    │
+│  │                    ▼                      │    │
+│  │    ┌───────────────────────────────┐      │    │
+│  │    │ Perform inference             │      │    │
+│  │    └───────────────┬───────────────┘      │    │
+│  │                    │                      │    │
+│  │                    ▼                      │    │
+│  │    ┌───────────────────────────────┐      │    │
+│  │    │ Output results                │      │    │
+│  │    └───────────────────────────────┘      │    │
+│  └───────────────────────────────────────────┘    │
+│                                                   │
+└───────────────────────────────────────────────────┘
+
+ExecuTorch runtime deployment to an embedded system +
+ +## Accept the Arm End User License Agreement + +```bash +export ARM_FVP_INSTALL_I_AGREE_TO_THE_CONTAINED_EULA=True +``` + +## Set Up the Arm Build Environment + +This example builds the [MobileNet V2](https://pytorch.org/hub/pytorch_vision_mobilenet_v2/) computer vision model. The model is a convolutional neural network (CNN) that extracts visual features from an image. It is used for image classification and object detection. The actual Python code for the MobileNet V2 model is in the `executorch` repo: [executorch/examples/models/mobilenet_v2/model.py](https://github.com/pytorch/executorch/blob/main/examples/models/mobilenet_v2/model.py). + +You can read a detail explanation of the build steps here: [ARM Ethos-U Backend](https://docs.pytorch.org/executorch/stable/backends-arm-ethos-u.html). + +1. Run the steps to set up the build environment: + + ```bash + ./examples/arm/setup.sh \ + --target-toolchain arm-none-eabi-gcc + ``` + +2. Update your environment: + ```bash + source examples/arm/ethos-u-scratch/setup_path.sh + ``` + +## Build the ExecuTorch .pte +Now you will build the `.pte` file, that will be used on the NXP board. + +1. Build the [MobileNet V2](https://pytorch.org/hub/pytorch_vision_mobilenet_v2/) ExecuTorch `.pte` runtime file using [aot_arm_compiler](https://github.com/pytorch/executorch/blob/2bd96df8de07bc86f2966a559e3d6c80fc324896/examples/arm/aot_arm_compiler.py): + + ```bash + python3 -m examples.arm.aot_arm_compiler \ + --model_name="mv2" \ + --quantize \ + --delegate \ + --debug \ + --target ethos-u55-256 + + ``` + +{{% notice Note %}} +| Flag | Meaning | +| ------------------------ | --------------------------------------------------- | +| `--model_name="mv2"` | Example model: MobileNetV2 (small, efficient) | +| `--quantize` | Enables int8 quantization (required for Ethos-NPUs) | +| `--delegate` | Enables offloading layers to the Ethos backend | +| `--debug` | Verbose build output | +| `--target ethos-u55-256` | Targets the Ethos-U55 | + +The `--quantize` flag uses one input example, so the resulting model will likely have poor classification performance. +{{% /notice %}} + +3. Check that the `mv2_arm_delegate_ethos-u55-256.pte` file was generated: + + ```bash + ls mv2_arm_delegate_ethos-u55-256.pte + ``` + +## Troubleshooting +**`setup.sh`** +- If you see the following error in the `setup.sh` output: + ```bash { output_lines = "1-2" } + Failed to build tosa-tools-v0.80 + ERROR: Could not build wheels for tosa-tools-v0.80, which is required to install pyproject.toml-based projects + ``` + then: + 1. Increase the swap space to 8 GB: + ```bash + fallocate -l 8G /swapfile + chmod 600 /swapfile + mkswap /swapfile + swapon /swapfile + ``` + - [optional] Deallocate the swap space after you complete this learning path: + ```bash + swapoff /swapfile + rm /swapfile + ``` + + {{% notice macOS %}} + Increase the "Memory Limit" in Docker settings to 12 GB: + ![Increase the "Memory Limit" in Docker settings to 12 GB alt-text#center](./increase-the-memory-limit-to-12-gb.jpg "Increase the Memory Limit in Docker settings to 12 GB") + + {{% /notice %}} + + 2. Re-run `setup.sh` + ```bash + ./examples/arm/setup.sh --i-agree-to-the-contained-eula + ``` + +- If you see the following error in the `setup.sh` output: + ```bash { output_lines = "1-2" } + Failed to build tosa-tools + ERROR: Failed to build installable wheels for some pyproject.toml based projects (tosa-tools) + ``` + then do the below troubleshooting steps. + 1. Install any missing build tools: + ```bash + apt update && apt install -y \ + cmake \ + build-essential \ + ninja-build \ + python3-dev \ + libboost-all-dev + ``` + 2. Re-run `setup.sh` + ```bash + ./examples/arm/setup.sh --i-agree-to-the-contained-eula + ``` +- If you see the following error in the `setup.sh` output: + ```bash { output_lines = "1-8" } + ... + ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. + tosa-tools 0.80.2.dev1+g70ed0b4 requires jsonschema, which is not installed. + tosa-tools 0.80.2.dev1+g70ed0b4 requires flatbuffers==23.5.26, but you have flatbuffers 24.12.23 which is incompatible. + tosa-tools 0.80.2.dev1+g70ed0b4 requires numpy<2, but you have numpy 2.3.1 which is incompatible. + ... + ``` + then just re-run `setup.sh` + ```bash + ./examples/arm/setup.sh --i-agree-to-the-contained-eula \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/9-build-executorch-runner-for-cm33.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/9-build-executorch-runner-for-cm33.md new file mode 100644 index 0000000000..de7f900cae --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/9-build-executorch-runner-for-cm33.md @@ -0,0 +1,251 @@ +--- +title: Build the executor_runner firmware +weight: 10 + +### FIXED, DO NOT MODIFY +layout: learningpathall +--- + +## Set up MCUXpresso for VS Code + +Install the MCUXpresso extension in VS Code: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +1. Open VS Code and press `Ctrl+Shift+X` to open Extensions +2. Search for "MCUXpresso for VS Code" +3. Click **Install** on the NXP extension +{{< /tab >}} +{{< tab header="macOS" >}} +1. Open VS Code and press `Cmd+Shift+X` to open Extensions +2. Search for "MCUXpresso for VS Code" +3. Click **Install** on the NXP extension +{{< /tab >}} +{{< /tabpane >}} + +Configure the ARM toolchain path: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +1. Open Settings with `Ctrl+,` +2. Search for **MCUXpresso: Toolchain** +3. Set the toolchain path to: `/opt/arm-gnu-toolchain-14.2.rel1-x86_64-arm-none-eabi/bin` +{{< /tab >}} +{{< tab header="macOS" >}} +1. Open Settings with `Cmd+,` +2. Search for **MCUXpresso: Toolchain** +3. Set the toolchain path to: `/opt/arm-gnu-toolchain-14.2.rel1-x86_64-arm-none-eabi/bin` +{{< /tab >}} +{{< /tabpane >}} + +Install the MCUXpresso SDK for FRDM-MIMX93: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +1. Open Command Palette: `Ctrl+Shift+P` +2. Type: **MCUXpresso: Install MCUXpresso SDK** +3. Search for "FRDM-MIMX93" or select **MCIMX93-EVK** +4. Select the latest SDK and click **Install** +{{< /tab >}} +{{< tab header="macOS" >}} +1. Open Command Palette: `Cmd+Shift+P` +2. Type: **MCUXpresso: Install MCUXpresso SDK** +3. Search for "FRDM-MIMX93" or select **MCIMX93-EVK** +4. Select the latest SDK and click **Install** +{{< /tab >}} +{{< /tabpane >}} + +{{% notice Note %}} +If the FRDM-MIMX93 development board is not listed in the current MCUXpresso SDK catalog, you can alternatively select **MCIMX93-EVK** as they share the same i.MX93 SoC with Cortex-M33 core architecture. The SDK compatibility ensures seamless development across both platforms. +{{% /notice %}} + +## Clone the executor_runner repository + +Clone the ready-to-build executor_runner project: + +```bash +git clone https://github.com/fidel-makatia/Executorch_runner_cm33.git +cd Executorch_runner_cm33 +``` + +The repository contains the complete runtime source code and build configuration for Cortex-M33. + +## Copy ExecuTorch libraries + +The executor_runner requires prebuilt ExecuTorch libraries with Ethos-U NPU support from your Docker container. + +Find your ExecuTorch build container: + +```bash { output_lines = "2-3" } +docker ps -a +CONTAINER ID IMAGE COMMAND CREATED STATUS +abc123def456 executorch "/bin/bash" 2 hours ago Exited +``` + +Copy the libraries: + +```bash +docker cp abc123def456:/home/ubuntu/executorch/cmake-out/lib/. ./executorch/lib/ +docker cp abc123def456:/home/ubuntu/executorch/. ./executorch/include/executorch/ +``` + +Replace `abc123def456` with your actual container ID. + +{{% notice Note %}} +In some Docker containers, the `cmake-out` folder might not exist. If you don't see the libraries, run the following command to build them: + +```bash +./examples/arm/run.sh --build-only +``` + +The libraries will be generated in `arm_test/cmake-out`. +{{% /notice %}} + +Verify the libraries: + +```bash { output_lines = "2-5" } +ls -lh executorch/lib/ +-rw-r--r-- 1 user user 2.1M libexecutorch.a +-rw-r--r-- 1 user user 856K libexecutorch_core.a +-rw-r--r-- 1 user user 1.3M libexecutorch_delegate_ethos_u.a +``` + +## Configure the project for FRDM-MIMX93 + +Open the project in VS Code: + +```bash +code . +``` + +Initialize the MCUXpresso project: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +1. Press `Ctrl+Shift+P` to open Command Palette +2. Type: **MCUXpresso: Import Repository** +3. Select the current folder +4. Choose **MIMX9352_cm33** as the target processor +{{< /tab >}} +{{< tab header="macOS" >}} +1. Press `Cmd+Shift+P` to open Command Palette +2. Type: **MCUXpresso: Import Repository** +3. Select the current folder +4. Choose **MIMX9352_cm33** as the target processor +{{< /tab >}} +{{< /tabpane >}} + +VS Code generates the MCUXpresso configuration. + +## Configure memory settings + +The Cortex-M33 has 108KB of RAM. The default memory configuration allocates: +- 16KB for the method allocator (activation tensors) +- 8KB for the scratch allocator (temporary operations) + +These settings are in `CMakeLists.txt`: + +```cmake +target_compile_definitions(${MCUX_SDK_PROJECT_NAME} PRIVATE + ET_ARM_BAREMETAL_METHOD_ALLOCATOR_POOL_SIZE=0x4000 # 16KB + ET_ARM_BAREMETAL_SCRATCH_TEMP_ALLOCATOR_POOL_SIZE=0x2000 # 8KB + ET_MODEL_PTE_ADDR=0x80100000 # DDR address for model +) +``` + +{{% notice Note %}} +If you see "region RAM overflowed" errors during build, reduce these pool sizes. For example, change to 0x2000 (8KB) and 0x1000 (4KB) respectively. +{{% /notice %}} + +## Build the firmware + +Configure the build system: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +1. Press `Ctrl+Shift+P` +2. Type: **CMake: Configure** +3. Select **ARM GCC** as the kit +4. Choose **Debug** or **Release** +{{< /tab >}} +{{< tab header="macOS" >}} +1. Press `Cmd+Shift+P` +2. Type: **CMake: Configure** +3. Select **ARM GCC** as the kit +4. Choose **Debug** or **Release** +{{< /tab >}} +{{< /tabpane >}} + +Build the project: + +Press `F7` or: + +{{< tabpane code=false >}} +{{< tab header="Windows/Linux" >}} +1. Press `Ctrl+Shift+P` +2. Type: **CMake: Build** +{{< /tab >}} +{{< tab header="macOS" >}} +1. Press `Cmd+Shift+P` +2. Type: **CMake: Build** +{{< /tab >}} +{{< /tabpane >}} + +Watch the build output: + +```output +[build] Scanning dependencies of target executorch_runner_cm33.elf +[build] [ 25%] Building CXX object source/arm_executor_runner.cpp.obj +[build] [ 50%] Building CXX object source/arm_memory_allocator.cpp.obj +[build] [ 75%] Linking CXX executable executorch_runner_cm33.elf +[build] [100%] Built target executorch_runner_cm33.elf +[build] Build finished with exit code 0 +``` + +Verify the build succeeded: + +```bash { output_lines = "2" } +ls -lh build/executorch_runner_cm33.elf +-rwxr-xr-x 1 user user 601K executorch_runner_cm33.elf +``` + +Check memory usage to ensure it fits in the Cortex-M33: + +```bash { output_lines = "2-3" } +arm-none-eabi-size build/executorch_runner_cm33.elf + text data bss dec hex filename + 52408 724 50472 103604 19494 executorch_runner_cm33.elf +``` + +The total RAM usage (data + bss) is approximately 51KB, well within the 108KB limit. + +## Troubleshooting + +**ARM toolchain not found:** + +Add the toolchain to your PATH: + +```bash +export PATH=/opt/arm-gnu-toolchain-14.2.rel1-x86_64-arm-none-eabi/bin:$PATH +``` + +**Cannot find ExecuTorch libraries:** + +Verify the libraries were copied correctly: + +```bash +ls executorch/lib/libexecutorch*.a +``` + +If missing, re-copy from the Docker container. + +**Region RAM overflowed:** + +Edit `CMakeLists.txt` and reduce the memory pool sizes: + +```cmake +ET_ARM_BAREMETAL_METHOD_ALLOCATOR_POOL_SIZE=0x2000 # 8KB +ET_ARM_BAREMETAL_SCRATCH_TEMP_ALLOCATOR_POOL_SIZE=0x1000 # 4KB +``` + +Then rebuild with `F7`. \ No newline at end of file diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_index.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_index.md new file mode 100644 index 0000000000..bee7a23b14 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_index.md @@ -0,0 +1,64 @@ +--- +title: Observing Ethos-U on a Physical Device, Built on Arm + +minutes_to_complete: 120 + +who_is_this_for: This is an introductory topic for developers and data scientists new to Tiny Machine Learning (TinyML), who want to observe ExecuTorch performance on a physical device. + +learning_objectives: + - Identify suitable physical Arm-based devices for TinyML applications. + - Optionally, configure physical embedded devices. + - Deploy a TinyML ExecuTorch model to NXP's FRDM i.MX 93 applicaiton processor (board). + +prerequisites: + - Purchase of a NXP [FRDM i.MX 93](https://www.nxp.com/design/design-center/development-boards-and-designs/frdm-i-mx-93-development-board:FRDM-IMX93) board. + - A USB Mini-B to USB Type-A cable, or a USB Mini-B to USB Type-C cable. + - Basic knowledge of Machine Learning concepts. + - A computer running Linux or macOS. + - VS Code + +author: Waheed Brown, Fidel Makatia Omusilibwa + +### Tags +skilllevels: Introductory +subjects: ML +armips: + - Cortex-A + - Cortex-M + - Ethos-U + +operatingsystems: + - Linux + - macOS + +tools_software_languages: + - Baremetal + - Python + - PyTorch + - ExecuTorch + - Arm Compute Library + - GCC + +further_reading: + - resource: + title: TinyML Brings AI to Smallest Arm Devices + link: https://newsroom.arm.com/blog/tinyml + type: blog + - resource: + title: Arm Machine Learning Resources + link: https://www.arm.com/developer-hub/embedded-and-microcontrollers/ml-solutions/getting-started + type: documentation + - resource: + title: Arm Developers Guide for Cortex-M Processors and Ethos-U NPU + link: https://developer.arm.com/documentation/109267/0101 + type: documentation + + + + +### FIXED, DO NOT MODIFY +# ================================================================================ +weight: 1 # _index.md always has weight of 1 to order correctly +layout: "learningpathall" # All files under learning paths have this same wrapper +learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. +--- diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_next-steps.md b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_next-steps.md new file mode 100644 index 0000000000..c3db0de5a2 --- /dev/null +++ b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/_next-steps.md @@ -0,0 +1,8 @@ +--- +# ================================================================================ +# FIXED, DO NOT MODIFY THIS FILE +# ================================================================================ +weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. +title: "Next Steps" # Always the same, html page title. +layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. +--- diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/imx-93-application-processor-soc.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/imx-93-application-processor-soc.png new file mode 100644 index 0000000000..838d47f6d5 Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/imx-93-application-processor-soc.png differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-swap-space-to-4-gb.jpg b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-swap-space-to-4-gb.jpg new file mode 100644 index 0000000000..bd36c08bac Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-swap-space-to-4-gb.jpg differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-the-memory-limit-to-12-gb.jpg b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-the-memory-limit-to-12-gb.jpg new file mode 100644 index 0000000000..441c20636b Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/increase-the-memory-limit-to-12-gb.jpg differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-board-built-in-ml-demos.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-board-built-in-ml-demos.png new file mode 100644 index 0000000000..e50d656b13 Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-board-built-in-ml-demos.png differ diff --git a/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-frdm-imx-93-board-soc-highlighted.png b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-frdm-imx-93-board-soc-highlighted.png new file mode 100644 index 0000000000..b50ace3a21 Binary files /dev/null and b/content/learning-paths/embedded-and-microcontrollers/observing-ethos-u-on-nxp/nxp-frdm-imx-93-board-soc-highlighted.png differ