Sprint 20-25 architecture plans — completing the roadmap

Sprint 20: Legacy code ingestion + modernization workflows Sprint 21: Semantic cross-language transpilation engine Sprint 22: Assembly languages (x86/ARM) + C++ remaining gaps Sprint 23: Architect mode — problem description to tech stack + skeleton Sprint 24: Security annotations, OWASP detection, secure transpilation Sprint 25: Self-hosting, end-to-end scenarios, polish, release prep Cumulative: ~508 steps, ~5000 tests, 19+ languages, 90+ MCP tools, 80+ annotation types across 10 subjects. Post-25 training data harvest builds on real workflow decisions accumulated across all sprints. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 05:19:22 +00:00
parent 4f5542a47f
commit 023809ef4b
7 changed files with 867 additions and 0 deletions
--- a/progress.md
+++ b/progress.md
@@ -903,6 +903,19 @@ inference-to-routing bridge replace training data export (deferred to post-25).
 - `sprint17_plan.md` — Languages: F#, VB.NET, SQL dialects (12 steps, ~136 tests)
 - `sprint18_plan.md` — Claude Code Plugin + MCP Workflow Tools (11 steps, ~124 tests, 75+ tools)
 - `sprint19_plan.md` — GUI Phase 2: Workflow Visualization (10 steps, ~112 tests)
+- `sprint20_plan.md` — Legacy Code Ingestion (11 steps, ~124 tests)
+- `sprint21_plan.md` — Cross-Language Transpilation Engine (11 steps, ~124 tests, 83+ tools)
+- `sprint22_plan.md` — Languages: Assembly + C++ remaining gaps (11 steps, ~124 tests)
+- `sprint23_plan.md` — Architect Mode + Tech Stack Selection (11 steps, ~124 tests, 87+ tools)
+- `sprint24_plan.md` — Security + Static Analysis (11 steps, ~124 tests, 90+ tools)
+- `sprint25_plan.md` — Integration, Self-Hosting, Polish (16 steps, ~180 tests)
+
+### Cumulative projections (Sprints 9-25):
+- **Steps:** ~508 (245-508)
+- **Tests:** ~5000+
+- **Languages:** 19+ parsers and generators
+- **MCP Tools:** 90+
+- **Annotation Types:** 80+ across 10 subjects

 ### Key architectural decisions:
 1. **Skeleton AST** = project specification before code exists (annotations on empty functions/classes)
--- a/sprint20_plan.md
+++ b/sprint20_plan.md
@@ -0,0 +1,127 @@
+# Sprint 20 Plan: Legacy Code Ingestion
+
+## Context
+
+This is what started the whole project. Take old code — code written before modern
+safety practices, before ownership models, before async/await, before anyone thought
+about annotation-driven workflows — and bring it into the modern world.
+
+Sprint 20 builds the ingestion pipeline: parse legacy code, infer what it does and
+what's dangerous about it, annotate it with the full taxonomy, and produce a modernization
+plan as a workflow. The output is a skeleton of the modernized version with annotations
+guiding the orchestrator.
+
+**Prerequisites delivered by earlier sprints:**
+- 17+ language parsers (Sprint 14, 17) including C, the primary legacy language
+- Annotation inference across all 8 subjects (Sprint 11e)
+- Routing annotations for workflow dispatch (Sprint 11e)
+- Workflow model + orchestration engine (Sprints 12, 15)
+- C++ depth for parsing complex legacy C++ (Sprints 12, 16)
+
+---
+
+## Phase 20a: Legacy Analysis Engine (Steps 438-443)
+
+### Step 438: Code Age + Idiom Detection (12 tests)
+- Detect legacy patterns: K&R function declarations, pre-C99 variable declarations,
+  manual memory management without RAII, raw pointer arithmetic, goto-heavy control flow,
+  deprecated API usage (gets, sprintf, strcpy)
+- Language version detection: C89/C99/C11, C++03/11/14/17/20, Java 6/8/11/17,
+  Python 2/3
+- Idiom scoring: how "legacy" is this code on a 0-10 scale
+- Output: per-function and per-file legacy score with specific findings
+
+### Step 439: Safety Audit via Annotations (12 tests)
+- Run annotation inference focused on safety:
+  - Buffer overflow risk: @BoundsCheck(unchecked) on array operations
+  - Use-after-free risk: @Owner(Manual) without clear @Lifetime tracking
+  - Null dereference: @Nullability(nullable) on unchecked pointers
+  - Race conditions: shared mutable state without @Sync or @Atomic
+  - Integer overflow: arithmetic without @Overflow annotation
+- Generate structured safety report with risk levels per function/file
+- Map findings to CWE (Common Weakness Enumeration) codes where applicable
+
+### Step 440: Modernization Suggestions (12 tests)
+- For each legacy pattern, suggest a modernization path:
+  - `malloc/free` → smart pointers (C++) or GC language (Java/Python/Rust)
+  - `sprintf` → `std::format` or safe string builders
+  - Raw loops → range-based for or iterators
+  - `goto` → structured control flow
+  - Manual vtable → virtual methods or trait objects
+  - Global mutable state → dependency injection or module pattern
+- Each suggestion is an annotation: @Modernize(from="pattern", to="replacement", risk="level")
+- Suggestions grouped by effort: quick wins vs deep refactors
+
+### Step 441: Modernization Workflow Generation (12 tests)
+- Given a legacy file with analysis + suggestions:
+  1. Create skeleton of modernized version (same structure, modern idioms)
+  2. Annotate skeleton with routing: quick wins → deterministic, complex → LLM, risky → human
+  3. Generate workflow from skeleton (WorkItems with dependencies)
+  4. Order: safe changes first, risky changes last (human review on risky)
+- Output: a ready-to-execute WorkflowState that the orchestrator can run
+
+### Step 442: Modernization RPC + MCP (12 tests)
+- `whetstone_analyze_legacy` — run legacy analysis on a file/project
+- `whetstone_suggest_modernization` — get modernization suggestions
+- `whetstone_create_modernization_workflow` — generate the full workflow
+- `whetstone_get_safety_report` — structured safety audit
+- 79+ MCP tools total
+
+### Step 443: Phase 20a Integration (8 tests)
+- Ingest a C file with legacy patterns → analyze → suggest → create workflow →
+  orchestrate → deterministic modernizations applied → complex items prepared for LLM
+- Safety report flags real issues (buffer overflow, null deref)
+- Modernization workflow respects risk ordering
+- Cross-language modernization: C → Rust (ownership safety)
+
+---
+
+## Phase 20b: Cross-Language Migration (Steps 444-448)
+
+### Step 444: Migration Plan Generator (12 tests)
+- Given source language + target language + codebase:
+  1. Analyze entire project structure (modules, dependencies, APIs)
+  2. Identify migration units (which files/modules migrate together)
+  3. Determine migration order (dependencies first, leaf modules first)
+  4. Annotate each unit: effort, risk, routing
+- Output: MigrationPlan with ordered phases
+
+### Step 445: API Boundary Preservation (12 tests)
+- When migrating a module, preserve its API surface:
+  - Function signatures maintained (or mapped to target-language equivalents)
+  - Public types preserved with cross-language compatibility notes
+  - FFI annotations where mixed-language boundaries exist
+  - @Contract annotations verify pre/post conditions match
+- Test: migrate internal implementation, verify external API unchanged
+
+### Step 446: Test Generation for Migration Validation (12 tests)
+- For each migrated module, auto-generate validation tests:
+  - Input/output equivalence tests (same inputs → same outputs)
+  - Edge case coverage from @Contract pre/post conditions
+  - Performance regression tests from @Complexity annotations
+- Tests generated as skeleton work items (routed to SLM/LLM)
+- Test language matches target language
+
+### Step 447: Migration Execution Integration (12 tests)
+- Hook migration plan into the orchestration engine:
+  - Each migration unit becomes a workflow
+  - Cross-unit dependencies respected in execution order
+  - Rollback annotations: if a unit fails validation, flag for human review
+- Progressive migration: some units migrated while others remain in source language
+- FFI boundary management during partial migration
+
+### Step 448: Phase 20b Integration + Sprint Summary (8 tests)
+- Full migration: 3-file C project → Rust, preserving API boundaries
+- Generated tests validate behavioral equivalence
+- Partial migration: 2 files migrated, 1 remains C with FFI annotations
+- Sprint 20 totals: legacy analysis, modernization, migration all operational
+
+---
+
+## Step & Test Summary
+
+| Phase | Steps | Tests | Theme |
+|-------|-------|-------|-------|
+| 20a | 438-443 | 68 | Legacy analysis, safety audit, modernization workflow |
+| 20b | 444-448 | 56 | Migration planning, API preservation, test generation |
+| **Total** | **438-448** | **~124** | 11 steps |
--- a/sprint21_plan.md
+++ b/sprint21_plan.md
@@ -0,0 +1,127 @@
+# Sprint 21 Plan: Cross-Language Transpilation Engine
+
+## Context
+
+Until now, "cross-language projection" has been syntactic: map AST nodes from
+source language conventions to target language conventions. Sprint 21 makes it
+semantic: the transpiler understands what code *does*, not just what it *looks like*,
+and produces behaviorally equivalent output in the target language.
+
+The key difference: syntactic projection maps `for i in range(10)` to `for (int i = 0; i < 10; i++)`.
+Semantic transpilation understands that this is "iterate 10 times" and selects the
+most idiomatic, efficient representation in the target: `(0..10).for_each(|i| ...)` in
+Rust, `(dotimes (i 10) ...)` in Common Lisp, `repeat(10) { ... }` in Kotlin.
+
+Annotations drive the translation: @Intent tells the transpiler what the code means,
+@Complexity guides how much to restructure, @Risk determines how conservative to be.
+
+---
+
+## Phase 21a: Semantic Translation Layer (Steps 449-454)
+
+### Step 449: Intent-Driven Translation (12 tests)
+- @Intent annotations on functions describe what they do, not how
+- Translation engine reads intent and selects target-idiomatic implementation:
+  - @Intent("sort list ascending") → Python: `sorted()`, Rust: `.sort()`, SQL: `ORDER BY`
+  - @Intent("find element by key") → Python: dict lookup, Java: HashMap.get, SQL: WHERE
+- When intent is clear + target has idiomatic equivalent → direct translation
+- When intent is ambiguous → preserve source structure + flag for review
+
+### Step 450: Algorithm-Level Equivalence (12 tests)
+- Recognize algorithmic patterns in source AST:
+  - Linear search → map to target's standard library search
+  - Sort implementations → map to target's standard library sort
+  - Map/filter/reduce patterns → map to target's functional idioms
+  - Builder patterns → map to target's construction idioms
+- Pattern library: 20+ common algorithmic patterns with cross-language equivalents
+- @Complexity annotation guides whether to use standard library or preserve manual implementation
+
+### Step 451: Type System Translation (12 tests)
+- Map type systems across languages with annotation guidance:
+  - Nullable types: Java Optional → Rust Option → Python Optional → C nullable pointer
+  - Generic types: Java List<T> → Rust Vec<T> → Python list[T] → C void* + size
+  - Enum types: Java enum → Rust enum → Python Enum → C enum/constants
+  - Class hierarchies: inheritance → traits/interfaces → protocols → struct+vtable
+- @TypeState and @Nullability annotations preserve type-safety intent across languages
+- Lossy translations (e.g., generics → void*) get @Risk(medium) annotation
+
+### Step 452: Concurrency Model Translation (12 tests)
+- Map concurrency patterns across language runtimes:
+  - Python async/await → Rust async/await (different runtime model!)
+  - Java threads → Go goroutines → Rust threads → C pthreads
+  - Python GIL-safe → Rust Send+Sync → Go channel-based → C mutex-based
+  - JavaScript Promises → Rust futures → Python asyncio
+- @Exec, @Sync, @ThreadModel annotations guide the translation
+- @Blocking annotation prevents mapping async → sync or vice versa without review
+
+### Step 453: Memory Model Translation (12 tests)
+- Translate ownership/lifecycle across memory models:
+  - Rust ownership → C++ smart pointers → C manual → Java/Python GC
+  - @Owner(Unique) → unique_ptr → malloc+single_free → (no equivalent, GC)
+  - @Owner(Shared_ARC) → shared_ptr → reference counting → GC
+  - @Lifetime(Scope) → RAII → stack allocation → try-finally
+- Loss of safety tracked: Rust → C loses borrow checker guarantees → @Risk(high)
+- Gain of safety tracked: C → Rust gains borrow checker → @Risk(lowered)
+
+### Step 454: Phase 21a Integration (8 tests)
+- Semantic transpilation: Python data processing → Rust (idiomatic, not line-by-line)
+- Algorithm recognition: Python bubble sort → Rust .sort() when @Intent("sort") present
+- Type system: Java generics → Rust generics (preserved), Java → C (void* with @Risk)
+- Concurrency: Python async → Rust async (different runtime acknowledged)
+- Memory: C manual → Rust ownership (safety gained, annotated)
+- Annotation guidance: @Intent + @Complexity + @Risk all influence translation choices
+
+---
+
+## Phase 21b: Verification + Confidence (Steps 455-459)
+
+### Step 455: Behavioral Equivalence Checking (12 tests)
+- For each transpiled function, generate equivalence assertions:
+  - Same inputs → same outputs (for pure functions)
+  - Same side effects (for stateful functions, best-effort)
+  - Same error conditions (exception/error mapping)
+- Express as @Contract annotations on the transpiled function
+- Confidence score: how sure are we the translation preserves behavior
+
+### Step 456: Transpilation Confidence Scoring (12 tests)
+- Score each translation 0.0–1.0 based on:
+  - Direct standard library mapping (0.95+)
+  - Algorithm pattern match (0.85+)
+  - Structural translation with @Intent (0.70+)
+  - Structural translation without @Intent (0.50)
+  - Unknown pattern (0.30 — flag for human review)
+- Route to review gate based on confidence threshold
+- Low-confidence translations get @Review(required, human) automatically
+
+### Step 457: Translation Report (12 tests)
+- Per-file and per-function translation report:
+  - Source construct → target construct mapping
+  - Confidence score
+  - Annotations applied/changed
+  - Safety delta (gained or lost)
+  - Idiomatic vs literal translation choice and why
+- Project-level summary: % idiomatic, % literal, % flagged for review
+
+### Step 458: Transpilation RPC + MCP (12 tests)
+- `whetstone_transpile` — transpile a function/file with semantic awareness
+- `whetstone_get_translation_report` — get translation details
+- `whetstone_verify_equivalence` — generate equivalence assertions
+- `whetstone_get_confidence` — confidence score for a translation
+- 83+ MCP tools total
+
+### Step 459: Phase 21b Integration + Sprint Summary (8 tests)
+- Full semantic transpilation: Python project → Rust with confidence scores
+- Translation report shows idiomatic choices with reasoning
+- Low-confidence translations auto-flagged for review
+- Equivalence assertions generated and meaningful
+- Sprint 21 totals: semantic transpilation operational for all 17+ languages
+
+---
+
+## Step & Test Summary
+
+| Phase | Steps | Tests | Theme |
+|-------|-------|-------|-------|
+| 21a | 449-454 | 68 | Intent-driven, algorithm, type, concurrency, memory translation |
+| 21b | 455-459 | 56 | Equivalence checking, confidence scoring, reports, MCP tools |
+| **Total** | **449-459** | **~124** | 11 steps |
--- a/sprint22_plan.md
+++ b/sprint22_plan.md
@@ -0,0 +1,119 @@
+# Sprint 22 Plan: Language Batch 3 — Assembly + Remaining Gaps
+
+## Context
+
+Assembly language is the floor. If Whetstone can represent assembly, it can
+represent anything. This sprint adds x86 and ARM assembly, fills remaining
+C++ gaps from feature-requests.md (items 12-17), and addresses any language
+coverage holes discovered during real-world use in Sprints 18-21.
+
+**After Sprint 22:** 19+ language parsers and generators, C++ self-hosting
+coverage at 90%+.
+
+---
+
+## Phase 22a: Assembly Language Support (Steps 460-465)
+
+Assembly is not like other languages. There's no abstraction — every instruction
+is an explicit machine operation. The AST representation must capture: registers,
+instructions, addressing modes, labels, directives, and sections.
+
+### Step 460: Assembly AST Nodes (12 tests)
+- New node types (all in `editor/src/ast/AssemblyNodes.h`):
+  - `AssemblyInstruction` — opcode, operands (register/immediate/memory)
+  - `AssemblyLabel` — name, isGlobal
+  - `AssemblyDirective` — type (.data, .text, .global, .section, .byte, .word)
+  - `AssemblyRegister` — name, size (8/16/32/64-bit)
+  - `AssemblyMemoryOperand` — base register, offset, index, scale
+- These live inside Function nodes (a function = label + instructions + ret)
+- Serialization for all new types
+
+### Step 461: x86 Assembly Parser (12 tests)
+- Intel and AT&T syntax support (mode flag)
+- Parse: instructions (mov, add, sub, cmp, jmp, call, ret, push, pop),
+  labels, directives (.section, .global, .data, .text), comments (;)
+- Register recognition: rax/eax/ax/al, rbx, rcx, ..., rsp, rbp, rip
+- Memory addressing: [rax], [rbp-8], [rax+rbx*4+8]
+- Sections (.text, .data, .bss) as NamespaceDeclaration equivalents
+
+### Step 462: ARM Assembly Parser (12 tests)
+- ARM/AArch64 instruction set
+- Parse: MOV, ADD, SUB, LDR, STR, B, BL, CMP, branch conditions
+- Registers: r0-r15 (ARM), x0-x30 (AArch64), sp, lr, pc
+- Addressing modes: [r0], [r0, #4], [r0, r1, LSL #2]
+- Directives: .global, .text, .data, .word, .align
+
+### Step 463: Assembly Generator — x86 + ARM (12 tests)
+- Generate valid assembly from AST
+- x86 generator: Intel syntax (default) or AT&T syntax (flag)
+- ARM generator: standard ARM syntax
+- Cross-architecture: x86 → ARM translation for simple instruction sequences
+- `commentPrefix() -> "; "` (x86) or `"@ "` (ARM)
+
+### Step 464: Assembly Annotation Mapping (12 tests)
+- @Exec(native) on all assembly functions
+- @Target(x86) or @Target(arm) — architecture annotation
+- @Align from .align directives
+- @Risk(high) on all assembly (inherently unsafe)
+- @Complexity from instruction count and branch density
+- Register pressure analysis → @Complexity annotation
+- Cross-language: C function → assembly (compiler-like lowering)
+- Cross-language: assembly → C (decompilation-like lifting)
+
+### Step 465: Phase 22a Integration (8 tests)
+- Parse real x86 assembly → valid AST
+- Parse real ARM assembly → valid AST
+- Cross-architecture: simple x86 → ARM translation
+- Assembly → C lifting for simple functions
+- C → assembly lowering annotations
+- Annotations meaningful for assembly
+- 19+ languages total
+- Assembly AST round-trips through Serialization
+
+---
+
+## Phase 22b: C++ Remaining Gaps (Steps 466-470)
+
+Feature-requests.md items 12-17: the constructs that come up in real C++ code
+but aren't architecturally complex — they just need parser/generator support.
+
+### Step 466: Range-Based For + Structured Bindings (12 tests)
+- `for (const auto& x : container)` → RangeForStatement node
+- `auto [key, value] = pair;` → StructuredBinding node
+- Parse, serialize, generate for both
+- Cross-language: range-for → Python for-in, Rust for-in, Java for-each
+
+### Step 467: Exception Handling (12 tests)
+- TryCatchStatement node: try block, catch clauses (each with type + variable), optional finally
+- ThrowExpression node
+- Noexcept specifier on functions
+- Parse `try { } catch (const exception& e) { } catch (...) { }`
+- Cross-language: C++ try/catch → Rust Result/match, Python try/except, Go if err
+
+### Step 468: Operator Overloading + Friend (12 tests)
+- OperatorOverload node: operator symbol, parameters, return type, isFriend
+- Parse `bool operator<(const Foo&) const;`
+- Parse `friend ostream& operator<<(ostream&, const Foo&);`
+- Cross-language: operators → named methods in other languages (Java compareTo, Python __lt__)
+
+### Step 469: Initializer Lists + STL Patterns (12 tests)
+- Initializer list expressions: `{1, 2, 3}`
+- STL container recognition: vector, map, set, string → annotated with type info
+- Iterator pattern detection → @Loop annotation
+- Parse `std::vector<int> v = {1, 2, 3};`
+
+### Step 470: Phase 22b Integration + Sprint Summary (8 tests)
+- C++ self-hosting coverage: parse TransformEngineExtended.h (inheritance, virtual)
+- All feature-requests items 1-17 now covered
+- Remaining self-hosting gaps identified (for Sprint 25 polish)
+- Sprint 22 totals: 19+ languages, C++ at 90%+ coverage
+
+---
+
+## Step & Test Summary
+
+| Phase | Steps | Tests | Theme |
+|-------|-------|-------|-------|
+| 22a | 460-465 | 68 | Assembly AST nodes, x86 parser, ARM parser, generators, annotations |
+| 22b | 466-470 | 56 | Range-for, exceptions, operators, initializer lists, STL |
+| **Total** | **460-470** | **~124** | 11 steps |
--- a/sprint23_plan.md
+++ b/sprint23_plan.md
@@ -0,0 +1,133 @@
+# Sprint 23 Plan: Architect Mode + Tech Stack Selection
+
+## Context
+
+The endgame feature: describe a problem in natural language, and the system
+proposes a technology stack, project structure, and annotated skeleton. This
+is the architect's power tool — not replacing the architect, but giving them
+a structured starting point they can review, modify, and approve.
+
+Sprint 23 builds on everything: the annotation taxonomy (Sprints 10-11),
+the skeleton AST (Sprint 11e), the workflow model (Sprint 12), the orchestrator
+(Sprint 15), the language coverage (17+ languages), and the transpilation engine
+(Sprint 21).
+
+---
+
+## Phase 23a: Problem Decomposition (Steps 471-476)
+
+### Step 471: Problem Description Parser (12 tests)
+- Accept natural language problem description
+- Extract structured requirements:
+  - Functional requirements (what it should do)
+  - Non-functional requirements (performance, security, scalability)
+  - Platform constraints (web, mobile, embedded, server, CLI)
+  - Integration requirements (databases, APIs, existing systems)
+- Output: StructuredRequirements JSON with confidence per extraction
+- This is the one place where LLM assistance is expected in the architecture —
+  the extraction is an MCP prompt template, not hardcoded NLP
+
+### Step 472: Module Decomposition Engine (12 tests)
+- Given StructuredRequirements, propose module structure:
+  - Identify major subsystems (auth, data, API, UI, etc.)
+  - Define module boundaries and interfaces
+  - Estimate complexity per module
+  - Identify cross-cutting concerns (logging, error handling, config)
+- Output: ModuleGraph with nodes (modules) and edges (dependencies)
+- Multiple decomposition strategies: microservice vs monolith vs layered
+
+### Step 473: Technology Stack Selector (12 tests)
+- Given requirements + module graph, recommend tech stack:
+  - Language selection per module (guided by requirements):
+    - Performance-critical → Rust or C++
+    - Web frontend → TypeScript
+    - Data processing → Python
+    - System scripting → Go or Python
+    - Database → SQL dialect based on requirements
+    - Embedded → C or assembly
+  - Framework suggestions (as annotations, not framework-specific code)
+  - Database selection based on data requirements
+- Configurable preferences: "I prefer Rust for backend", "must use PostgreSQL"
+- Output: TechStackDecision with reasoning per choice
+
+### Step 474: Skeleton Generation from Requirements (12 tests)
+- Given module graph + tech stack:
+  1. Create skeleton modules in chosen languages
+  2. Create skeleton functions/classes with @Intent annotations from requirements
+  3. Add routing annotations: @Complexity, @ContextWidth, @Automatability
+  4. Add contract annotations: @Contract from functional requirements
+  5. Add dependency annotations between modules
+- Output: Multi-file skeleton project ready for workflow creation
+
+### Step 475: Architect Review Interface (12 tests)
+- Present the proposed stack + skeleton to the architect for review:
+  - Tech stack summary with reasoning
+  - Module dependency graph (visual in GUI, structured in MCP)
+  - Per-module: language choice, complexity estimate, key annotations
+  - Modification tools: change language, merge modules, add modules, adjust routing
+- Architect approves → skeleton becomes the project spec
+- Architect modifies → re-run affected selections
+
+### Step 476: Phase 23a Integration (8 tests)
+- Full flow: "Build a REST API for a bookstore with user auth, PostgreSQL,
+  and a React frontend" → structured requirements → module decomposition →
+  tech stack (TypeScript frontend, Python/Rust backend, PostgreSQL) →
+  skeleton project → architect review → approved → workflow created
+- 87+ MCP tools total
+
+---
+
+## Phase 23b: Project Templates + Scaffolding (Steps 477-481)
+
+### Step 477: Architecture Templates (12 tests)
+- Pre-built architecture patterns:
+  - **REST API:** routes + handlers + models + middleware + database
+  - **CLI Tool:** argument parsing + commands + output formatting
+  - **Library:** public API + internal modules + tests + documentation stubs
+  - **Microservice:** service + transport + storage + health check
+  - **Full Stack:** frontend + backend + database + deployment config
+- Templates are parameterized: entity names, field definitions, auth method
+- Each template produces a fully-annotated skeleton project
+
+### Step 478: Scaffold File Generation (12 tests)
+- Generate actual project files on disk from approved skeleton:
+  - Source files with skeleton code + Semanno annotations
+  - Project configuration (package.json, Cargo.toml, pyproject.toml, CMakeLists.txt)
+  - Directory structure following language conventions
+  - .gitignore appropriate for selected stack
+  - .whetstone/ directory with sidecar files and workflow state
+- Uses existing fileCreate/fileWrite MCP infrastructure
+
+### Step 479: Dependency + Build System Awareness (12 tests)
+- Annotate dependencies between modules with build system info:
+  - Python: imports and pip requirements
+  - Rust: crate dependencies in Cargo.toml format
+  - Node: npm packages in package.json format
+  - C++: include paths and CMake targets
+  - SQL: migration ordering
+- Not full build system integration — annotation-level awareness for context
+
+### Step 480: Multi-Language Project Orchestration (12 tests)
+- Workflow creation for multi-language projects:
+  - Each module may be a different language
+  - Cross-language interfaces annotated with @Link and @Shim
+  - Build order respects cross-language dependencies
+  - FFI boundaries explicitly annotated for human review
+- Orchestrator handles multi-language workflow as a single project
+
+### Step 481: Phase 23b Integration + Sprint Summary (8 tests)
+- REST API template → scaffold files on disk → workflow → orchestrate
+- Multi-language project: Python service + Rust core + SQL schema
+- Architect modifies tech stack mid-planning → skeleton regenerates
+- All scaffolded files have proper annotations and structure
+- Sprint 23 totals: architect mode operational, templates working
+
+---
+
+## Step & Test Summary
+
+| Phase | Steps | Tests | Theme |
+|-------|-------|-------|-------|
+| 23a | 471-476 | 68 | Problem decomposition, tech stack selection, skeleton generation |
+| 23b | 477-481 | 56 | Templates, scaffolding, build awareness, multi-language orchestration |
+| **Total** | **471-481** | **~124** | 11 steps |
--- a/sprint24_plan.md
+++ b/sprint24_plan.md
@@ -0,0 +1,135 @@
+# Sprint 24 Plan: Security + Static Analysis
+
+## Context
+
+Security isn't a feature you bolt on at the end — but it is a domain that
+benefits from having the full annotation taxonomy, transpilation engine,
+and orchestration system in place first. Sprint 24 adds security as a
+first-class concern: security annotations, vulnerability detection, dependency
+audit, and security review as a routing category.
+
+When the system encounters `@Risk(security)` or detects a vulnerable pattern,
+it knows to route that work item to human review with full security context.
+The annotation system makes security findings *actionable* — not just warnings
+in a log, but structured routing decisions.
+
+---
+
+## Phase 24a: Security Annotations + Detection (Steps 482-487)
+
+### Step 482: Security Annotation Types — Subject 10 (12 tests)
+- New annotation subject: Security
+  - `TrustBoundaryAnnotation` — marks where trusted/untrusted data crosses
+  - `InputValidationAnnotation` — validation: "sanitized" | "raw" | "escaped"
+  - `AuthAnnotation` — authRequired: bool, roles: vector, method: "jwt" | "session" | "api-key"
+  - `EncryptionAnnotation` — algorithm, keySize, mode (at-rest, in-transit)
+  - `SecretsAnnotation` — marks code that handles secrets, rotation policy
+  - `CORSAnnotation` — origins, methods, credentials
+- Serialization, CompactAST, SidecarPersistence for all new types
+
+### Step 483: OWASP Pattern Detection (12 tests)
+- Detect OWASP Top 10 vulnerability patterns via AST analysis:
+  - **Injection:** string concatenation in SQL queries, unsanitized user input in commands
+  - **Broken Auth:** hardcoded credentials, missing auth checks on routes
+  - **Sensitive Data:** secrets in source code, unencrypted storage
+  - **XXE:** XML parsing without entity restriction
+  - **Broken Access Control:** missing role checks, direct object references
+  - **Security Misconfiguration:** debug mode in production annotations, permissive CORS
+  - **XSS:** unsanitized output in HTML generation
+  - **Deserialization:** untrusted data deserialization
+  - **Known Vulnerabilities:** deprecated crypto, known-insecure functions
+  - **Logging:** sensitive data in log statements
+- Each detection produces structured diagnostic (E1300+ range)
+- Confidence level per finding (some are heuristic, not certain)
+
+### Step 484: Dependency Audit Annotations (12 tests)
+- `DependencyAnnotation` — package name, version, known CVEs, license
+- Parse dependency files: requirements.txt, Cargo.toml, package.json, pom.xml
+- Flag known-vulnerable versions from annotation data
+- License compatibility checking (GPL vs MIT vs Apache in same project)
+- Not a real-time CVE database — annotation-based tracking that the
+  orchestration engine can route for periodic review
+
+### Step 485: Security-Aware Routing (12 tests)
+- Security findings automatically set routing annotations:
+  - OWASP finding → @Risk(security) + @Review(required, human)
+  - Trust boundary crossing → @ContextWidth(project) (need full context)
+  - Crypto code → @Automatability(human) (never auto-generate crypto)
+  - Auth logic → @Review(required, human) + @Ambiguity(high)
+- Routing engine recognizes security annotations as highest-priority review triggers
+- Security review items get enhanced context: threat model, adjacent trust boundaries
+
+### Step 486: Security Report Generation (12 tests)
+- Structured security report for a project:
+  - Executive summary: risk level, critical findings count
+  - Per-finding: OWASP category, CWE code, affected code, confidence, remediation
+  - Trust boundary map: which modules trust which
+  - Annotation coverage: % of code with security annotations
+- Export formats: JSON (for orchestration), Markdown (for human review)
+
+### Step 487: Phase 24a Integration (8 tests)
+- Scan a Python web app → detect SQL injection + missing auth → security report
+- Security annotations route sensitive code to human review
+- OWASP findings produce structured diagnostics
+- Dependency audit flags known-vulnerable packages
+- Security report includes trust boundary analysis
+- MCP tools: whetstone_security_scan, whetstone_security_report, whetstone_audit_dependencies
+- 90+ MCP tools total
+
+---
+
+## Phase 24b: Secure Transpilation (Steps 488-492)
+
+### Step 488: Security-Preserving Translation (12 tests)
+- When transpiling code with security annotations:
+  - @InputValidation preserved in target language (not silently dropped)
+  - @TrustBoundary maintained at module boundaries
+  - @Auth annotations mapped to target-language auth patterns
+  - @Encryption mapped to target-language crypto libraries
+- Security annotations NEVER auto-removed during transpilation
+- Any security annotation that can't map to target → @Review(required, human)
+
+### Step 489: Secure-by-Default Code Generation (12 tests)
+- Generated code follows secure defaults:
+  - SQL queries use parameterized queries, never string concatenation
+  - User input always has @InputValidation(raw) until explicitly sanitized
+  - File operations include path traversal checks
+  - Network calls include timeout and TLS verification
+  - Crypto uses current best practices (no MD5, no SHA1 for security)
+- Deterministic/template workers enforce these defaults
+- LLM context bundles include security requirements
+
+### Step 490: Threat Model Integration (12 tests)
+- Annotations can express a threat model:
+  - @TrustBoundary(name, level) — "external-api", "database", "user-input"
+  - Data flow through trust boundaries → automatic @InputValidation requirements
+  - Missing validation at boundary → E1300 diagnostic
+- Threat model as a project-level annotation (on Module)
+- Visualization data for Sprint 19's GUI (trust boundary overlay)
+
+### Step 491: Security Testing Skeleton Generation (12 tests)
+- Generate security test skeletons alongside production code:
+  - Input validation tests (fuzzing boundaries)
+  - Auth bypass tests
+  - SQL injection probe tests
+  - XSS payload tests
+  - Access control matrix tests
+- Skeletons annotated with @Intent describing what to test
+- Routed: simple validation tests → SLM, complex penetration → human
+
+### Step 492: Phase 24b Integration + Sprint Summary (8 tests)
+- Transpile Python → Rust: security annotations preserved and strengthened
+- Generated code uses parameterized SQL, validated input
+- Threat model produces trust boundary diagnostics
+- Security test skeletons generated and meaningful
+- Sprint 24 totals: security is a first-class orchestration concern
+
+---
+
+## Step & Test Summary
+
+| Phase | Steps | Tests | Theme |
+|-------|-------|-------|-------|
+| 24a | 482-487 | 68 | Security annotations, OWASP detection, dependency audit, routing |
+| 24b | 488-492 | 56 | Secure transpilation, defaults, threat models, security tests |
+| **Total** | **482-492** | **~124** | 11 steps |
--- a/sprint25_plan.md
+++ b/sprint25_plan.md
@@ -0,0 +1,213 @@
+# Sprint 25 Plan: Integration, Self-Hosting, Polish
+
+## Context
+
+The capstone. Every system built in Sprints 9-24 comes together. Sprint 25 has
+three objectives:
+
+1. **Self-hosting:** Whetstone parses, annotates, and transpiles its own source code.
+   This is the ultimate integration test — if the tool can understand itself, it can
+   understand anything.
+
+2. **End-to-end validation:** Every workflow path is tested with real-world scenarios.
+   Not unit tests of individual components, but full journeys: problem description →
+   annotated skeleton → orchestrated workflow → transpiled output → security review →
+   deployed artifact.
+
+3. **Polish:** Performance optimization, edge case cleanup, documentation for the
+   MCP tool surface, and preparation for the post-25 training data phase.
+
+---
+
+## Phase 25a: Self-Hosting (Steps 493-498)
+
+### Step 493: Parse Full Whetstone Codebase (12 tests)
+- Feed every .h file in editor/src/ and editor/src/ast/ through the C++ parser
+- Track: successful parse rate (target: 95%+), constructs recognized, constructs skipped
+- For each file: verify function count, class count, annotation node count
+- Log all unparseable constructs for future language depth work
+- This is the audit: how complete is our C++ coverage really?
+
+### Step 494: Annotate Whetstone's Own Code (12 tests)
+- Run AnnotationInference on parsed AST of Whetstone's source
+- Expected findings:
+  - @Complexity scores for each major class
+  - @Risk annotations on unsafe patterns (raw pointers, void casts)
+  - @Owner annotations on unique_ptr/shared_ptr usage
+  - @Intent annotations where function names are descriptive enough
+  - @TailCall on recursive visitors
+- Validate: inferred annotations are actually correct for our code
+
+### Step 495: Transpile Whetstone Modules (12 tests)
+- Pick 3 self-contained Whetstone headers and transpile:
+  - `AnnotationConflictExtended.h` → Python (structs → classes, simple mapping)
+  - `ResponseBudget.h` → Rust (ownership maps cleanly)
+  - `Icons.h` → Java (constants → enum class)
+- Verify: transpiled code is structurally correct, annotations preserved
+- Measure: translation confidence scores, how much was idiomatic vs literal
+
+### Step 496: Self-Annotated Workflow (12 tests)
+- Create a workflow for Whetstone's own code modernization:
+  - Target: modernize AnnotationValidator.h (add the validation rules it's missing)
+  - Skeleton: functions for each new validation rule
+  - Route: simple rules → deterministic, complex rules → LLM, new diagnostic codes → human review
+- Orchestrate: verify deterministic rules generate valid code
+- This is Whetstone improving itself through its own workflow system
+
+### Step 497: Self-Hosting Metrics Report (12 tests)
+- Comprehensive self-hosting report:
+  - Parse coverage: X/Y files, Z% of constructs
+  - Annotation accuracy: how many inferred annotations are correct
+  - Transpilation quality: confidence scores across target languages
+  - Workflow viability: can we actually use the system to develop itself
+- Gap analysis: what's still missing for 100% self-hosting
+- Comparison: metrics vs Sprint 16's initial self-hosting attempt (progress delta)
+
+### Step 498: Phase 25a Integration (8 tests)
+- Full self-hosting pipeline: parse → annotate → transpile → workflow → validate
+- Metrics show 95%+ parse coverage
+- At least 3 headers transpile to other languages
+- Self-improvement workflow runs and produces valid output
+- This is the thesis proven: Whetstone annotates and works on itself
+
+---
+
+## Phase 25b: End-to-End Scenarios (Steps 499-503)
+
+### Step 499: Scenario — Greenfield Project (12 tests)
+- Full journey: "Build a URL shortener with user accounts"
+  - Problem decomposition → tech stack (Python backend, PostgreSQL, TypeScript frontend)
+  - Skeleton generation → annotated modules
+  - Workflow creation → orchestration
+  - Deterministic tasks auto-completed (CRUD endpoints, model definitions)
+  - LLM tasks prepared with context bundles
+  - Human review items surfaced for auth logic
+  - Security scan on generated code
+  - Final project structure on disk
+
+### Step 500: Scenario — Legacy Modernization (12 tests)
+- Full journey: modernize a C codebase
+  - Ingest legacy C code → safety audit → modernization suggestions
+  - Migration plan → Rust as target language
+  - Workflow with mixed routing: simple functions deterministic, complex logic LLM
+  - API boundary preservation verified
+  - Security annotations strengthened (C manual memory → Rust ownership)
+  - Migration tests generated
+
+### Step 501: Scenario — Cross-Language Port (12 tests)
+- Full journey: port a Python ML pipeline to Rust for performance
+  - Parse Python source → annotate → identify hot loops
+  - Semantic transpilation: algorithm-level translation
+  - Memory model translation: Python GC → Rust ownership
+  - Concurrency translation: Python asyncio → Rust tokio patterns
+  - Confidence scoring across translation
+  - Human review for low-confidence translations
+  - Equivalence assertions generated
+
+### Step 502: Scenario — Multi-Model Orchestration (12 tests)
+- Full journey: a 20-function project using all worker types
+  - 5 deterministic tasks (getters, setters, constructors) → auto-complete
+  - 5 template tasks (CRUD, simple transforms) → auto-complete
+  - 5 SLM tasks (straightforward logic) → prepared with narrow context
+  - 3 LLM tasks (complex logic) → prepared with wide context
+  - 2 human tasks (architectural decisions) → flagged for review
+  - Progress tracking throughout
+  - Cost estimation vs actual comparison
+  - All review gates exercised
+
+### Step 503: Phase 25b Integration (8 tests)
+- All 4 scenarios run without errors
+- Each scenario exercises a different workflow path
+- Cost tracking accurate across scenarios
+- Security scan produces findings on generated code
+- Event stream captures full journey for each scenario
+- No regressions across the entire test suite (steps 245-503)
+
+---
+
+## Phase 25c: Polish + Release Prep (Steps 504-508)
+
+### Step 504: Performance Optimization (12 tests)
+- Profile the full pipeline on large projects (50+ files, 500+ functions)
+- Optimize hot paths:
+  - AST serialization/deserialization
+  - Annotation inference (batch processing)
+  - Context assembly (caching for shared project context)
+  - Compact AST generation
+- Target: <100ms per function for parse + annotate + route
+- Memory: no memory leaks in headless mode under sustained use
+
+### Step 505: MCP Tool Documentation (12 tests)
+- Every MCP tool has:
+  - Clear description (what it does, when to use it)
+  - Input schema with examples
+  - Output schema with field descriptions
+  - Error codes and their meanings
+- Tool categories documented: AST, Diagnostics, Workflow, Routing, Security, etc.
+- Generate tool documentation as a resource the MCP client can read
+
+### Step 506: Edge Case Cleanup (12 tests)
+- Handle gracefully:
+  - Empty files, binary files, extremely large files
+  - Malformed annotations, circular dependencies
+  - Concurrent workflow modifications
+  - Interrupted workflows (save + resume)
+  - Unknown language (fallback to plain text)
+  - MCP protocol errors (malformed requests)
+- Each edge case produces a meaningful error, not a crash
+
+### Step 507: Full Test Suite Regression (12 tests)
+- Run the complete test suite: steps 245-506
+- Expected: 5000+ tests all passing
+- No flaky tests (run 3 times, all pass)
+- Performance benchmark: full suite completes in <60 seconds
+- Memory baseline: no leaks across full suite
+
+### Step 508: Sprint 25 Summary + Post-25 Readiness (8 tests)
+- Final metrics:
+  - 19+ language parsers and generators
+  - 90+ MCP tools
+  - 80+ annotation types across 10 subjects
+  - 500+ steps, 5000+ tests
+  - Self-hosting at 95%+ coverage
+  - 4 end-to-end scenarios validated
+- Post-25 readiness:
+  - Event stream data accumulated (ready for training data harvest)
+  - Workflow decisions logged (routing outcomes, review decisions)
+  - Transpilation pairs logged (source → target with confidence)
+  - System ready for Sprint 26: Training Data Harvest + Fine-Tuning Pipeline
+
+---
+
+## Step & Test Summary
+
+| Phase | Steps | Tests | Theme |
+|-------|-------|-------|-------|
+| 25a | 493-498 | 68 | Self-hosting: parse, annotate, transpile, workflow on own code |
+| 25b | 499-503 | 56 | End-to-end scenarios: greenfield, legacy, port, multi-model |
+| 25c | 504-508 | 56 | Performance, docs, edge cases, regression, release readiness |
+| **Total** | **493-508** | **~180** | 16 steps |
+
+---
+
+## The Numbers
+
+**Across Sprints 9-25:**
+- **Steps:** ~508 (245-508)
+- **Tests:** ~5000+
+- **Languages:** 19+ parsers and generators
+- **MCP Tools:** 90+
+- **Annotation Types:** 80+ across 10 subjects
+- **New AST Node Types:** 30+
+- **Sprint Plans:** 17 detailed plan documents
+
+**The system, completed:**
+- Universal AST with semantic annotations
+- Annotation-driven workflow orchestration (human/AI routing)
+- Cross-language semantic transpilation
+- Professional GUI with workflow visualization
+- MCP-first agent integration (model-agnostic)
+- Legacy code ingestion and modernization
+- Security-first with OWASP detection and secure-by-default generation
+- Self-hosting: the tool understands and works on itself
+- Ready for training data harvest from real workflow decisions