|
1 | 1 | # pcb_fault_detection_ui
|
2 | 2 |
|
3 |
| -A new Flutter project. |
| 3 | +A Flutter desktop application that allows you to find defects in PCB by using their images. |
4 | 4 |
|
5 |
| -## Getting Started |
| 5 | +This project leverages Flutter for GUI and Rust for the ML inference logic, utilizing the capabilities of the [flutter_rust_bridge](https://github.com/fzyzcjy/flutter_rust_bridge) framework. |
6 | 6 |
|
7 |
| -This project is a starting point for a Flutter application. |
| 7 | +It uses 2 YOLOv11 models: |
8 | 8 |
|
9 |
| -A few resources to get you started if this is your first Flutter project: |
| 9 | +- One to detect defects in the PCB copper tracks. |
| 10 | +- Another to detect components soldered onto the PCB track. |
10 | 11 |
|
11 |
| -- [Lab: Write your first Flutter app](https://docs.flutter.dev/get-started/codelab) |
12 |
| -- [Cookbook: Useful Flutter samples](https://docs.flutter.dev/cookbook) |
| 12 | +The two models are mutually exclusive, i.e. if an image is select as a track image, then it cannot also be selected as a component image. This is because a components image leads to many false positives when run on the track model. |
13 | 13 |
|
14 |
| -For help getting started with Flutter development, view the |
15 |
| -[online documentation](https://docs.flutter.dev/), which offers tutorials, |
16 |
| -samples, guidance on mobile development, and a full API reference. |
| 14 | +The copper track defect model executed directly on selected images (after slicing). |
17 | 15 |
|
18 |
| -## Using Rust Inside Flutter |
| 16 | +The components are first detected on the image of a known good benchmark PCB. Then images of other PCBs are selected which are compared with the benchmark to find defects. This also uses slicing. |
19 | 17 |
|
20 |
| -This project leverages Flutter for GUI and Rust for the backend logic, |
21 |
| -utilizing the capabilities of the |
22 |
| -[Rinf](https://pub.dev/packages/rinf) framework. |
| 18 | +## Slicing |
23 | 19 |
|
24 |
| -To run and build this app, you need to have |
25 |
| -[Flutter SDK](https://docs.flutter.dev/get-started/install) |
26 |
| -and [Rust toolchain](https://www.rust-lang.org/tools/install) |
27 |
| -installed on your system. |
28 |
| -You can check that your system is ready with the commands below. |
29 |
| -Note that all the Flutter subcomponents should be installed. |
| 20 | +The images are sliced into smaller square sub-images and then sent to the model for inference, as this enables it to detect small components/track defects. The results are then stitched together to get the final result. This is called [Slicing Aided Hyper Inference (SAHI)](https://ieeexplore.ieee.org/document/9897990). Much of the code for this has been translated from [python SAHI implementation](https://github.com/obss/sahi), but translated to Rust. |
30 | 21 |
|
31 |
| -```shell |
32 |
| -rustc --version |
33 |
| -flutter doctor |
34 |
| -``` |
| 22 | +## Why are you using Rust, and where are you using it? |
35 | 23 |
|
36 |
| -You also need to have the CLI tool for Rinf ready. |
| 24 | +Here, we use Rust to actually execute the ML model inference using the Rust [ort 2.0.0-rc.10](https://ort.pyke.io/) library. The models are stored in the ONNX format, which is compatible with Ort, as it internally uses the ONNX Runtime library. |
37 | 25 |
|
38 |
| -```shell |
39 |
| -cargo install rinf_cli |
40 |
| -``` |
| 26 | +Also, we use Rust, and the [image](https://docs.rs/image/latest/image/) and [ndarray](https://docs.rs/ndarray/latest/ndarray/) libraries for the image slicing. |
41 | 27 |
|
42 |
| -Signals sent between Dart and Rust are implemented using signal attributes. |
43 |
| -If you've modified the signal structs, run the following command |
44 |
| -to generate the corresponding Dart classes: |
| 28 | +Finally, we use the Rust [rayon](https://docs.rs/rayon/latest/rayon/) library, for easy & simple parallelism. It allows us to directly & easily run parallel operation on streams of data like (map, filter_map, flatten, sort, etc.) by using it's parallel iterators. |
45 | 29 |
|
46 |
| -```shell |
47 |
| -rinf gen |
48 |
| -``` |
| 30 | +Note: We have only compiled Ort with DirectML support as the only execution provider, along with the CPU fallback of course. The reason for this is because the CUDA binaries needlessly inflated the final application, when most devices we will demonstrate it on will not have a CUDA compatible GPU (including the PC that this was developed on). So, for the sake of simplicity, other execution providers are not included. However, the Ort library makes it very easy to add execution providers as necessary, so if needed they can be added. The code for the using them is also included, but commented out. |
49 | 31 |
|
50 |
| -Now you can run and build this app just like any other Flutter projects. |
| 32 | +## Demonstration |
51 | 33 |
|
52 |
| -```shell |
53 |
| -flutter run |
54 |
| -``` |
55 |
| - |
56 |
| -For detailed instructions on writing Rust and Flutter together, |
57 |
| -please refer to Rinf's [documentation](https://rinf.cunarist.com). |
| 34 | +View the demo at https://youtu.be/tCxNRT4C0cI |
0 commit comments