Add experimental design workflow documentation files in .exp/

Author: gauravagerwala

.exp/design-workflow-1-cargo-clippy.md

@@ -0,0 +1,103 @@
+# High-Level Design of Workflow #1: cargo-clippy
+
+## Overview
+
+The \"cargo-clippy\" workflow enables running Clippy as a Cargo subcommand (`cargo clippy`) to lint Rust code in Cargo projects, including workspaces and optionally dependencies. It catches common mistakes, suggests improvements, and can auto-fix issues via `--fix`. 
+
+This workflow wraps Cargo's `check` or `fix` commands, injecting Clippy into the Rust compiler pipeline by setting `RUSTC_WORKSPACE_WRAPPER` to `clippy-driver` and passing Clippy-specific arguments via `CLIPPY_ARGS`. During compilation, Clippy's lints are registered and executed at various stages (AST, HIR, MIR), emitting diagnostics based on configuration from CLI, `clippy.toml`, and code attributes.
+
+Inputs include project directory (Cargo.toml), flags (e.g., `--all-targets`, `--fix`), env vars (e.g., `RUSTFLAGS`), and config files. Outputs are lint messages, potential source modifications, and exit codes.
+
+Entry point: `src/main.rs` (cargo-clippy binary).
+
+## Components
+
+- **cargo-clippy (`src/main.rs`)**: Handles CLI parsing, determines subcommand (`check` or `fix` based on `--fix`), constructs environment variables, and spawns `cargo` process.
+
+- **clippy-driver (`src/driver.rs`)**: Custom Rust compiler driver extending `rustc_driver`. Uses `ClippyCallbacks` to:
+  - Load configuration.
+  - Register Clippy lints with the `LintStore`.
+  - Adjust compiler options (e.g., `mir_opt_level = 0` for unoptimized MIR analysis, disable certain passes).
+  - Track dependencies (e.g., `clippy.toml`, `CLIPPY_ARGS`) for Cargo rebuild triggers.
+
+- **clippy_lints**: Core library implementing ~750 lints. 
+  - `declared_lints::LINTS`: Array of all lint metadata.
+  - Modular implementations in submodules (e.g., `methods/`, `misc_early/`, `operators/`).
+  - `register_lint_passes(store, conf)`: Registers `rustc_lint::LintPass` instances, filtered by config.
+
+- **clippy_config**: Parses hierarchical configuration from `clippy.toml` files, inline attributes (`#[allow(clippy::lint)]`), and CLI flags to set lint levels (allow, warn, deny, forbid) and groups.
+
+- **clippy_utils**: Utilities for lint authors, including AST/HIR/MIR queries, type comparisons, symbol matching, etc., to share common logic.
+
+- **rustc_tools_util**: Provides version information, environment setup helpers, and utilities for interacting with Rustc internals.
+
+- **declare_clippy_lint**: Macro crate for declaring lint structs with metadata (name, level, description, etc.).
+
+## Sequence Diagram
+
+The following Mermaid sequence diagram illustrates the primary flow of information and control in the cargo-clippy workflow:
+
+```mermaid
+sequenceDiagram
+    participant U as User
+    participant CC as "cargo-clippy"
+    participant C as cargo
+    participant CD as "clippy-driver"
+    participant RI as rustc_interface
+    participant LS as "Lint Store"
+    participant CL as "Clippy Lints (clippy_lints)"
+    participant Config as "clippy_config"
+
+    U->>CC: cargo clippy [options]
+    Note right of CC: Parse args, set mode (check/fix)<br/>Set env vars: RUSTC_WORKSPACE_WRAPPER, CLIPPY_ARGS
+    CC->>C: cargo check/fix [user args]
+    C->>CD: Invoke as rustc for each compilation unit
+    CD->>RI: rustc_driver::run_compiler(args, ClippyCallbacks)
+    RI->>CD: Callbacks::config(config)
+    CD->>Config: load_conf() -> clippy.toml, attributes
+    Config-->>CD: Conf (enabled lints, levels)
+    Note right of CD: Adjust opts: mir_opt_level=0,<br/>disable some MIR passes, etc.
+    CD->>LS: register_lints(sess, store)
+    LS->>CL: Register declared_lints::LINTS
+    CL-->>LS: Lint metadata registered
+    LS->>CL: clippy_lints::register_lint_passes(store, conf)
+    CL-->>LS: LintPass instances registered (filtered by conf)
+    Note over RI: Compilation pipeline executes:<br/>- Parse to AST<br/>- Expand/lower to HIR<br/>- MIR construction<br/>- At each phase, applicable lints run via visitors/checkers
+    alt --fix mode
+        Note over CL,RI: Lints emit rustfix suggestions
+        RI->>C: Suggestions via compiler interface
+        C->>U: cargo fix applies to source files
+    else check mode
+        RI->>U: Print diagnostics to stdout/stderr
+    end
+```
+
+## Configuration and Customization
+
+- **Loading**: In `clippy-driver`, `clippy_config::Conf::read()` aggregates config from:
+  - Project-local `clippy.toml`.
+  - Ancestor directories.
+  - Code attributes.
+  - CLI flags passed via `CLIPPY_ARGS` (e.g., `-W clippy::perf`).
+- **Application**: Used to filter and level lints during registration. Supports groups (correctness, style, perf, etc.) and MSRV specifications.
+- **Tracking**: Config files and env vars are added to `ParseSess`'s depinfo for incremental builds.
+
+## Lint Execution Details
+
+- **Phases**: Lints are categorized and run at specific compiler stages:
+  - Early (pre-type-check): AST visitors in `misc_early/`.
+  - Post-type-check: HIR/MIR analyses in other modules.
+- **Implementation**: Each lint is a struct implementing `rustc_lint::LintPass` (or early variant), defining `get_lints()` and visitor/check methods.
+- **Diagnostics**: Emit via `sess.emit_warning()`, `emit_error()`, etc., with spans, explanations, and optional fixes.
+- **Fixes**: Many lints integrate with `rustfix` for `--fix` mode, generating edit suggestions applied by `cargo fix`.
+
+## Additional Design Aspects
+
+- **Wrapper Mode Handling**: `clippy-driver` detects if invoked as `rustc` wrapper (via Cargo) and adjusts args. Supports direct invocation or delegation (`--rustc`).
+- **Optimization Trade-offs**: Disables optimizations like MIR passes and format_args flattening for accurate linting, potentially increasing compile time.
+- **Skip Conditions**: Bypasses Clippy for Cargo metadata queries, cap-lints=allow (unless forcing Clippy), or irrelevant packages in --no-deps mode.
+- **Error Reporting**: Integrates with rustc's diagnostics; ICEs report to GitHub issues with version info.
+- **Development Features**: Tracks driver executable in debug mode for rebuilds during development.
+- **Dependencies**: Relies on `rustc_private` for compiler internals; must sync with Rust releases.
+
+This design provides tight integration with the Rust toolchain, enabling powerful static analysis while maintaining Cargo compatibility.

.exp/design-workflow-2-clippy-driver.md

@@ -0,0 +1,112 @@
+# High-Level Design of the clippy-driver Workflow
+
+## Overview
+
+The clippy-driver workflow enables direct invocation of Clippy as a Rust compiler driver, providing an interface similar to \`rustc\` for linting individual Rust source files (.rs) or crates in non-Cargo environments, such as custom build scripts, IDE integrations, or standalone analysis tools. It serves as the core execution engine behind \`cargo clippy\` (via the \`RUSTC_WRAPPER\` environment variable) but can be run independently.
+
+Key characteristics:
+- **Inputs**: Rust source files, rustc-compatible CLI arguments (e.g., \`--edition\`, \`--target\`), optional sysroot, configuration via \`clippy.toml\` or env vars like \`CLIPPY_ARGS\`.
+- **Outputs**: Lint diagnostics (warnings, errors, suggestions) emitted to stdout/stderr, optional compiled artifacts (e.g., binaries, LLVM IR) if compilation proceeds.
+- **Entry Point**: \`src/driver.rs\` – processes arguments, decides on Clippy activation, and orchestrates the compilation via \`rustc_driver\`.
+- **Core Mechanism**: Acts as a drop-in replacement for \`rustc\`, customizing the compiler session to register Clippy's lints, adjust optimization levels (e.g., MIR opt level 0 for precise analysis), and execute analysis passes during compilation phases.
+- **Decision Logic**: Skips Clippy lints if all are capped to \`allow\` or in certain Cargo info queries; falls back to plain \`rustc\` behavior.
+- **Dependencies**: Leverages \`clippy_lints\` for lint implementations, \`clippy_utils\` for analysis helpers, and \`clippy_config\` for settings.
+
+This workflow ensures deep integration with Rust's compiler pipeline, allowing lints to access internal representations (AST, HIR, MIR, TyCtxt) for sophisticated checks impossible in external tools.
+
+## Components
+
+- **Argument Processing (src/driver.rs)**: Parses CLI args using \`rustc_driver::args::raw_args\`, handles special modes (\`--rustc\` delegates to rustc, \`--help\`/\`--version\`), injects Clippy-specific flags (e.g., \`--cfg clippy\`) from \`CLIPPY_ARGS\`.
+- **Callbacks (ClippyCallbacks / RustcCallbacks)**: Implement \`rustc_driver::Callbacks\`; customize config (lint registration, unstable opts), session creation (dependency tracking for Cargo rebuilds, e.g., Cargo.toml, clippy-driver exe).
+- **Lint Registry (clippy_lints/src/lib.rs)**: \`register_lint_passes\` populates \`rustc_lint::LintStore\` with:
+  - Deprecated/renamed/removed lints.
+  - Pre-expansion passes (e.g., attribute collection).
+  - Early passes (AST/HIR-based, e.g., formatting, doc checks).
+  - Late passes (typed analysis, e.g., type checks, method calls).
+  - Supports conditional enabling via \`Conf\`.
+- **Configuration Loader (clippy_config)**: \`Conf::read\` parses \`clippy.toml\`, inline attributes (\`#[allow(clippy::lint)]\`), CLI overrides; influences lint levels and params.
+- **Lint Utilities (clippy_utils)**: Reusable functions for lints (e.g., type queries, def path matching, expression analysis).
+- **Rustc Infrastructure**: \`rustc_driver\`, \`rustc_interface\`, \`rustc_session\` for session management, diagnostics; \`rustc_lint\` for pass registration.
+- **Development Helpers**: Tracks files/envs for incremental builds; ICE hook for bug reporting to GitHub.
+
+## Main Execution Flow
+
+The primary sequence illustrates invocation, mode decision, and compiler orchestration.
+
+```mermaid
+sequenceDiagram
+    participant U as User
+    participant D as clippy-driver
+    participant RD as rustc_driver
+    participant C as Compiler Config & Callbacks
+    participant LS as Lint Store
+    participant CP as Compilation Pipeline
+    participant L as Lint Passes
+
+    U->>D: Invoke with args (e.g., file.rs [options])
+    D->>D: Parse args, handle --help/--version/--rustc
+    alt Clippy enabled (no cap-lints=allow, etc.)
+        D->>D: Add --cfg clippy, CLIPPY_ARGS
+        D->>RD: run_compiler(args, ClippyCallbacks)
+        RD->>C: config() - load conf, set opts (mir_opt_level=0)
+        C->>LS: register_lint_passes() - add early/late lints
+        RD->>C: psess_created() - track deps (Cargo.toml, etc.)
+        RD->>CP: Run phases (parse, expand, typeck, MIR, ...)
+        loop During phases
+            CP->>L: Execute registered passes (EarlyLintPass, LateLintPass, ...)
+            L->>CP: Emit diagnostics if issues found
+        end
+    else Rustc mode
+        D->>RD: run_compiler(args, RustcCallbacks)
+        RD->>CP: Standard compilation without Clippy lints
+    end
+    CP->>D: Output diagnostics, artifacts
+    D->>U: Print to stdout/stderr, exit code
+```
+
+## Lint Registration and Execution Flow
+
+Clippy registers passes tailored to compiler phases; execution occurs interleaved with compilation.
+
+### Registration Details
+- **Deprecated Handling**: Registers renames/removals for backward compatibility.
+- **Early Passes**: ~50+ implementations for syntax/style checks (e.g., \`DoubleParens\`, \`Formatting\`).
+- **Late Passes**: ~300+ for semantic analysis (e.g., \`Types\`, \`Methods\`, \`Loops\`), using \`TyCtxt\` for context.
+- **Shared State**: Some passes share storage (e.g., \`FormatArgsStorage\`) across lints.
+
+### Execution Sequence
+```mermaid
+sequenceDiagram
+    participant P as Compilation Pipeline
+    participant LS as Lint Store
+    participant EP as Early Passes
+    participant LP as Late Passes
+
+    Note over P: Parsing & Expansion Phase
+    P->>LS: Run pre-expansion passes
+    LS->>EP: Execute EarlyLintPass (e.g., Attributes)
+    EP->>P: Potential diagnostics
+
+    Note over P: Post-Expansion (AST/HIR)
+    P->>LS: Run early passes
+    LS->>EP: EarlyLintPass (e.g., formatting, doc checks)
+    EP->>P: Emit diagnostics
+
+    Note over P: Type Checking & MIR
+    P->>LS: Run late passes
+    LS->>LP: LateLintPass with TyCtxt (e.g., types, methods, loops)
+    LP->>P: Emit diagnostics via session
+```
+
+## Other Relevant High-Level Design Aspects
+
+- **Performance Considerations**: MIR opt level set to 0 avoids optimizations that could obscure bugs (e.g., constant folding); disables specific passes like \`CheckNull\`. Trade-off: Slower compilation for accuracy.
+- **Incremental Compilation**: Tracks runtime-accessed files (Cargo.toml, clippy.toml, driver exe) in \`file_depinfo\` and envs (CLIPPY_CONF_DIR, CLIPPY_ARGS) for Cargo-triggered rebuilds.
+- **Configuration Flexibility**: Hierarchical (CLI > file > attributes); supports lint groups, custom params (e.g., max nesting depth).
+- **Fallback and Compatibility**: Supports plain rustc delegation for queries or suppressed lints; handles arg files (\`@path\`), sysroot env.
+- **Extensibility**: Lints implement traits like \`EarlyLintPass\` or \`LateLintPass\`; new ones added via dev tools, auto-registered in \`lib.rs\`.
+- **Diagnostics**: Leverages rustc's system for structured output (spans, suggestions via rustfix); colorized via anstream.
+- **Limitations**: Tied to rustc version (via \`rustc_private\`); requires nightly for dev; no built-in fix mode (use cargo clippy --fix).
+- **Testing Integration**: Used in UI tests, dogfooding; ensures lints trigger correctly in direct mode.
+
+This design draws from source analysis of \`src/driver.rs\`, \`clippy_lints/src/lib.rs\`, and cross-references with project architecture in \`project-overview.md\`. For implementation details, see relevant crates' READMEs and Rustc driver docs.