create repo context

Zonglin Peng · meta-codesync[bot] · commit 7e0c7d044a78 · 2025-10-14T08:45:25.000-07:00
Differential Revision: D84163926
diff --git a/.llms/rules/Executorch.md b/.llms/rules/Executorch.md
@@ -0,0 +1,361 @@
+---
+oncalls:
+  - executorch
+llms-gk: devmate_executorch_cadence_md
+apply_to_regex: ".*"
+---
+
+# ExecuTorch: On-Device AI Inference Runtime
+
+## Project Description
+
+ExecuTorch is Meta's end-to-end solution for enabling on-device AI inference across mobile and edge devices including wearables, embedded systems, and microcontrollers. It extends PyTorch 2's compiler and export functionality to efficiently deploy models to edge devices with minimal memory footprint and superior performance. ExecuTorch powers Meta's on-device AI experiences across Facebook, Instagram, Meta Quest, Ray-Ban Meta Smart Glasses, and WhatsApp.
+
+**Key Value Propositions:**
+- **Portability**: Compatible across platforms from high-end mobile to constrained embedded systems
+- **Productivity**: Unified toolchain from PyTorch authoring through deployment
+- **Performance**: Lightweight runtime (<50kB core) with full hardware acceleration support
+
+**Supported Models**: LLMs (Llama, Qwen, Phi), Computer Vision, ASR, TTS, and multimodal models
+
+## Architecture Overview
+
+### Three-Phase Workflow
+1. **Program Preparation (AOT)**: Model export, quantization, memory planning, backend delegation
+2. **Runtime Preparation**: Program loading and initialization
+3. **Program Execution**: Kernel dispatch and inference
+
+### Four-Dialect Lowering Pipeline
+```
+PyTorch Model → torch.export()
+  ↓ ATen Dialect (PyTorch-compatible operators)
+  ↓ Core ATen Dialect (Decomposed fundamental ops)
+  ↓ Edge Dialect (Edge-optimized, dtype-specialized)
+  ↓ Backend Dialect (Hardware-specific optimizations)
+  ↓ .pte file (Serialized execution graph)
+```
+
+### Core Components
+- **Runtime (`xplat/executorch/runtime/`)**: Core C++ execution engine, tensor data structures, operator dispatch
+- **EXIR (`xplat/executorch/exir/`)**: Export IR system, four-dialect compilation pipeline
+- **Kernels (`xplat/executorch/kernels/`)**: Portable, optimized, and quantized operator implementations
+- **Backends (`xplat/executorch/backends/`)**: Hardware acceleration (Apple CoreML/MPS, Qualcomm QNN, ARM Ethos-U, Vulkan, XNNPACK, etc.)
+- **Extensions (`xplat/executorch/extension/`)**: LLM framework, training, module wrappers, platform integration
+- **DevTools (`xplat/executorch/devtools/`)**: ETDump profiling, ETRecord debugging, inspector APIs
+
+## Directory Structure & Dependencies
+
+```
+xplat/executorch/
+├── runtime/           # Core C++ runtime (<50kB, portable)
+│   ├── core/         # Fundamental types (Tensor, EValue, Error)
+│   ├── executor/     # Program loading and execution
+│   ├── kernel/       # Operator registration and dispatch
+│   ├── backend/      # Backend delegate APIs
+│   └── platform/     # Platform abstraction layer (PAL)
+├── exir/             # Export IR compiler (4-dialect pipeline)
+├── kernels/          # Operator implementations
+│   ├── portable/     # Pure C++ reference kernels
+│   ├── optimized/    # Hardware-optimized versions
+│   └── quantized/    # Quantization kernels
+├── backends/         # Hardware acceleration (15+ backends)
+│   ├── apple/        # CoreML, MPS
+│   ├── qualcomm/     # QNN (HTP/NPU)
+│   ├── arm/          # Ethos-U, TOSA
+│   ├── xnnpack/      # Optimized CPU
+│   └── vulkan/       # Cross-platform GPU
+├── extension/        # Runtime extensions
+│   ├── llm/          # LLM tokenizers, samplers, KV cache
+│   ├── module/       # Simplified C++ APIs
+│   └── android/ios/  # Platform integration
+├── devtools/         # Profiling and debugging tools
+├── examples/         # Model export and usage examples
+├── schema/           # FlatBuffer file format definitions
+└── tools/            # Build system integration
+```
+
+**Key Dependencies:**
+- **Internal**: PyTorch 2.10.0 (pinned via `torch_pin.py`)
+- **External**: FlatBuffers, numpy>=2.0.0, pybind11, CMake>=3.29
+- **Platform-Specific**: coremltools (Apple), QNN SDK (Qualcomm), ARM Compute Library
+
+**Build Systems**:
+- Buck2 (internal Meta builds)
+- CMake (OSS builds, primary)
+- Python setuptools (pip packages)
+
+## Build and Development Workflow
+
+### Quick Start
+```bash
+# Clone and setup
+git clone -b viable/strict https://github.com/pytorch/executorch.git
+cd executorch
+conda create -yn executorch python=3.10.0 && conda activate executorch
+
+# Install in development mode
+./install_executorch.sh --editable
+
+# Run tests
+pytest
+```
+
+### CMake Build Options
+```bash
+# Platform-specific builds (via presets)
+cmake .. --preset android-arm64-v8a  # Android
+cmake .. --preset ios                 # iOS
+cmake .. --preset macos               # macOS
+cmake .. --preset linux               # Linux
+
+# Feature flags
+-DEXECUTORCH_BUILD_XNNPACK=ON        # XNNPACK backend
+-DEXECUTORCH_BUILD_COREML=ON         # CoreML backend
+-DEXECUTORCH_BUILD_EXTENSION_LLM=ON  # LLM support
+-DEXECUTORCH_BUILD_TESTS=ON          # Build tests
+-DEXECUTORCH_ENABLE_LOGGING=ON       # Runtime logging
+-DCMAKE_BUILD_TYPE=Release           # Release build (required for perf)
+
+# Build and run
+cmake --build cmake-out -j9
+cmake --build cmake-out --target test
+```
+
+### Export and Run Example
+```bash
+# Export model
+python -m examples.portable.scripts.export --model_name="add"
+
+# Execute
+./cmake-out/executor_runner --model_path add.pte
+```
+
+### Build Scripts
+- `xplat/executorch/scripts/build_android_library.sh` - Android native builds
+- `xplat/executorch/scripts/build_apple_frameworks.sh` - iOS/macOS frameworks
+- `xplat/executorch/install_executorch.sh` - Main installation script
+
+### Cleaning and Rebuilding
+```bash
+./install_executorch.sh --clean
+git submodule sync && git submodule update --init --recursive
+```
+
+## Documentation References
+
+**Core Documentation** (`xplat/executorch/docs/`):
+- Architecture guides and design documents
+- Backend integration tutorials
+- Platform-specific deployment guides
+- API references and examples
+
+**README Files**:
+- `xplat/executorch/README.md` - Project overview and getting started
+- `xplat/executorch/CONTRIBUTING.md` - Contribution guidelines
+- Backend-specific READMEs in `xplat/executorch/backends/*/README.md`
+
+**Internal Resources**:
+- Workplace groups and internal wikis for Meta-specific deployment patterns
+- Oncall: `executorch` (check TARGETS files for component-specific oncalls)
+
+## Testing Strategy
+
+### Test Frameworks
+- **Python**: pytest with hypothesis for property-based testing
+- **C++**: GoogleTest for unit tests
+- **Backend Testing**: Modular test suite in `xplat/executorch/backends/test/`
+
+### Test Organization
+```
+xplat/executorch/test/      # Core runtime tests
+xplat/executorch/backends/test/  # Backend validation suite
+xplat/executorch/examples/models/test/  # End-to-end model tests
+```
+
+### Testing Patterns
+- **Unit Tests**: Individual operator and component validation
+- **Integration Tests**: End-to-end model export and execution
+- **Backend Tests**: Stage-based pipeline (Export → Quantize → ToEdge → Partition → Serialize)
+- **Numerical Validation**: Configurable tolerance (atol=1e-1, rtol=4e-2 for backends)
+- **Performance Testing**: ETDump profiling with timing and memory metrics
+
+### Running Tests
+```bash
+# Python tests
+pytest                           # All tests
+pytest backends/xnnpack/test     # Specific backend
+pytest -k "test_name"            # Specific test
+
+# C++ tests
+cmake --build cmake-out --target test
+
+# Backend validation
+python -m backends.test.test_runner
+```
+
+### Test Configuration
+- `xplat/executorch/pytest.ini` - pytest configuration with flake detection (50 reruns)
+- `xplat/executorch/Test.cmake` - C++ test definitions
+- CI/CD in `xplat/executorch/oss/.github/workflows/` - Multi-platform CI validation
+
+## Integration Points
+
+### PyTorch Integration
+- **torch.export()**: Primary model capture mechanism
+- **ATen Operators**: Compatible with PyTorch operator set
+- **Python Bindings**: `executorch.runtime` module for torch tensor integration
+- **Tensor Parsers**: Support for ATen, exec_aten, and portable tensor modes
+
+### Backend Delegation
+- **BackendInterface**: Standard contract for backend implementations
+- **Delegate API**: `executorch_call_delegate` for offloading subgraphs
+- **Partitioning**: Automatic or manual delegation of operations to accelerators
+- **External Tensors**: Support for backend-managed memory
+
+### Platform Integration
+- **Android**: JNI bindings in `xplat/executorch/extension/android/`
+- **iOS**: Swift package and frameworks in `xplat/executorch/extension/apple/`
+- **Embedded**: Zephyr RTOS support, bare-metal ARM builds
+- **Cross-Platform**: Vulkan for GPU acceleration across desktop and mobile
+
+### Data Flow
+```
+PyTorch Model (Python)
+  ↓ torch.export()
+EXIR Graph
+  ↓ Backend Partitioning
+Delegated Subgraphs + Portable Ops
+  ↓ Serialization
+.pte File (FlatBuffer)
+  ↓ Runtime Loading
+C++ Runtime
+  ↓ Execution
+CPU/GPU/NPU Hardware
+```
+
+## Developer Rules
+
+### Portable C++ Programming
+**CRITICAL for `xplat/executorch/runtime/` code:**
+- **No stdlib dependencies**: No `std::vector`, `std::string`, `std::unique_ptr`, `std::shared_ptr`
+- **No exceptions/RTTI**: No `throw`, `try/catch`, `dynamic_cast`, `typeid`
+- **No standard I/O**: No `printf()`, `cout`, `malloc()`, `new`, files, threads
+- **Use provided APIs**: `MemoryManager` for allocation, `ET_LOG` for logging, `Result<T>` for errors
+- **C++17 only**: Target C++17 standard, use `<cstdint>` types (`uint32_t`, `int64_t`)
+
+### Code Style
+**C++ (Google Style with modifications):**
+- **Column Limit**: 80 characters
+- **Indentation**: 2 spaces, no tabs
+- **Naming**:
+  - Classes/Types: `PascalCase` (e.g., `BackendDelegate`, `EValue`)
+  - Functions/Methods: `snake_case()` (differs from Google)
+  - Variables: `snake_case` (private members: `trailing_underscore_`)
+  - Files: `snake_case.cpp`, `snake_case.h`
+- **Includes**: Use `<angle brackets>` for all includes, sort alphabetically within groups
+- **Documentation**: Doxygen style (`/** ... */` for multi-line, `/// ...` for single-line)
+
+**Python (PEP 8 + Google Style):**
+- **Line Length**: 80 characters
+- **Type Hints**: Required throughout
+- **Naming**:
+  - Classes: `UpperCamelCase`
+  - Functions: `snake_case()` (private: `_leading_underscore()`)
+  - Constants: `UPPER_SNAKE_CASE`
+- **Docstrings**: Google style with type annotations
+- **Imports**: Standard library → typing → third-party → local
+
+### Error Handling
+**C++**: Use `runtime::Result<T>` and `runtime::Error` patterns
+```cpp
+Result<int> foo() {
+  ET_CHECK_OR_RETURN_ERROR(condition, error_msg);
+  // ...
+  return value;
+}
+// Caller:
+auto result = foo();
+ET_CHECK_OR_RETURN_ERROR(result.ok(), "Failed");
+int value = ET_UNWRAP(result);
+```
+
+**Python**: Use custom exceptions with `ExportErrorType` categorization
+```python
+from executorch.exir.error import InternalError, ExportError
+
+if not valid:
+    raise InternalError(ExportErrorType.INVALID_INPUT, "Description")
+```
+
+### Testing Requirements
+- All new features and bug fixes MUST include tests
+- Local testing: `pytest` (Python), `test/run_oss_cpp_tests.sh` (C++)
+- CI must pass before merge
+- Backend tests use relaxed tolerances (atol=1e-1, rtol=4e-2)
+
+### API Lifecycle
+- **Experimental**: Mark with `ET_EXPERIMENTAL` (C++) or `@experimental` (Python) - can change without notice
+- **Stable**: Default - breaking changes require deprecation
+- **Deprecated**: Mark with `ET_DEPRECATED`/`@deprecated` - remove after 2 minor releases
+- Document migration paths for deprecated APIs
+
+### Linting and Formatting
+**Setup and Run:**
+```bash
+# Setup (once)
+lintrunner init
+
+# Run all linters
+lintrunner -a
+
+# Auto-fix Python formatting
+ufmt format .
+```
+
+**Configured Tools:**
+- **Python**: black, flake8, ufmt, mypy, torchfix
+- **C++**: clang-format (18.1.3)
+- **CMake**: cmakelang
+
+### Common Pitfalls
+
+**Export Issues:**
+- **Custom Operators**: Register with `torch.library` BEFORE export
+- **Dynamic Shapes**: Specify bounds with `dynamic_shapes` parameter
+- **Unsupported Ops**: Check operator support - not all PyTorch ops are supported
+
+**Runtime Issues:**
+- **Backend Linking**: Use `--whole-archive` when linking backend libraries
+- **Duplicate Registration**: Link only ONE `gen_operators_lib` per application
+- **Input Shape Mismatch**: Ensure runtime inputs match export-time example shapes
+- **Missing Operators**: Verify selective builds include all required ops
+
+**Performance Issues:**
+- **Debug Builds**: Always use `CMAKE_BUILD_TYPE=Release` for performance testing
+- **No Backend Delegation**: Ensure models use appropriate backends (XNNPACK, CoreML, etc.)
+- **Thread Count**: Optimize thread pool (often cores/2 or 4 on mobile)
+
+**Build Issues:**
+- **Python Dev Packages**: Install `python-dev` or `python3-devel`
+- **Submodule Sync**: Run `git submodule update --init` after pulling
+- **Clean Builds**: Use `--clean` flag if seeing unexpected build errors
+
+### Code Review Best Practices
+- Add clear PR title and description
+- Include test instructions and validation steps
+- Add reviewers based on CODEOWNERS or file blame
+- Label with appropriate release note tags
+- Use "Squash and merge" when ready
+
+### Performance Optimization
+- **Memory Planning**: Use AOT memory planning for constrained devices
+- **Quantization**: Apply QAT or PTQ for model size and performance
+- **Backend Selection**: Choose appropriate backend for target hardware
+- **Profiling**: Use `EXECUTORCH_SCOPE_PROF` macros and ETDump analysis
+- **Selective Build**: Link only required operators to minimize binary size
+
+### Additional Resources
+- **Contributing**: See `xplat/executorch/CONTRIBUTING.md` for detailed guidelines
+- **Oncall**: Primary oncall is `executorch`, component-specific in TARGETS files
+- **Code Ownership**: Check `xplat/executorch/CODEOWNERS` for file ownership
+- **CI Workflows**: See `xplat/executorch/oss/.github/workflows/` for CI configuration
