docs: Add comprehensive implementation plan and task list for feature 001

2026-02-03 16:14:58 -07:00
parent 0aa0841a4d
commit 9bd9e6b7ef
2 changed files with 1316 additions and 0 deletions
--- a/features/001-core-ast-structure/plan.md
+++ b/features/001-core-ast-structure/plan.md
@@ -0,0 +1,469 @@
 ---
 feature_number: 001
 feature_name: Core AST Structure
 status: Planning
 created_at: 2026-02-03T16:45:00Z
 spec_version: 1.0
 ---
 # Implementation Plan: Core AST Structure for SemAnno
 ## Technical Context
 **Technology Stack**:
 - **Workbench**: JetBrains MPS 2024.3+
 - **Target Language**: SemAnno (language definition in MPS)
 - **Model Format**: MPS XML (.mps files)
 - **Development Method**: Incremental concept definition with immediate editor testing
 **Key Decisions**:
 - All concepts defined in single SemAnno language module (no separate WhetstoneCore module)
 - Editors built in SemAnno.editor.mps with projection support for Python/C++ (added in future features)
 - No code generation or external tools required for this feature
 - Type system uses reference composition for nested type structures (ListType contains reference to inner Type)
 **Dependencies**:
 - MPS 2024.3 devkit for language design
 - None on external libraries or frameworks
 - Builds on existing SemAnno structure model (assumed to exist from previous session)
 ## Constitution Check
 **Project Constitution Principles** (from `.specswarm/constitution.md` or assumed defaults):
 - Clarity over cleverness: ✅ All concepts have clear, intuitive semantics
 - Consistency: ✅ Naming conventions and patterns applied uniformly
 - Type safety: ✅ Strong typing via MPS reference system
 - Testability: ✅ Manual AST creation is the test mechanism
 - Documentation: ✅ Specification documents design rationale
 **No conflicting principles detected** - proceeding with implementation.
 ## Implementation Phases
 ### Phase 0: Research & Exploration
 **Objectives**: Resolve any ambiguities, document assumptions, identify edge cases
 **Tasks**:
 1. **Research Task 1: MPS Editor Design Patterns**
   - Question: How are projection-ready editors structured in MPS for multi-projection languages?
   - Goal: Understand patterns used in multi-view languages (e.g., JetBrains MPS itself with structure/interface views)
   - Deliverable: Quick reference for editor cell patterns that support projection switching
   - Owner: Implementation team (research before Phase 1)
 2. **Research Task 2: Nested Type Composition in MPS**
   - Question: Best practices for building deeply nested type structures (list[map[string, optional[int]]])
   - Goal: Ensure type references are properly composable without circular dependencies
   - Deliverable: Sample MPS structure showing 3-4 levels of type nesting
   - Owner: Implementation team
 3. **Research Task 3: Annotation Attachment Patterns**
   - Question: How do MPS languages attach multiple annotations to a single concept?
   - Goal: Ensure annotations are optional, ordered, and independently removable
   - Deliverable: Example annotation structure from existing MPS language (e.g., jetbrains.mps.lang.core)
   - Owner: Implementation team
 **Status**: Research items are light and can be completed during Phase 1 implementation (not blocking)
 ---
 ### Phase 1: Core Structure Definition
 **Objectives**: Define all 30+ AST concepts in SemAnno.structure.mps
 **Deliverables**:
 - SemAnno.structure.mps with all concepts defined (Module, Function, Variable, all Statement types, all Expression types, all Type types, annotation concepts)
 - Preliminary constraints model (SemAnno.constraints.mps) with basic validation rules
 - Test model file (e.g., `languages/SemAnno/tests/SimpleExample.mps`) demonstrating a complete AST
 **Key Subtasks**:
 #### 1.1: Root & Container Concepts (1-2 hours)
 - Module concept with properties (name: string) and references (annotations, variables, functions)
 - Marker: "Container concepts complete"
 #### 1.2: Function & Parameter Concepts (2-3 hours)
 - Function concept: name, parameters[], returnType, body[], annotations
 - Parameter concept: name, type, defaultValue?
 - Establish parameter ordering and type reference patterns
 - Marker: "Function signature complete"
 #### 1.3: Variable Concept (1 hour)
 - Variable concept: name, type, initializer?, annotations
 - Ensure type and initializer reference logic
 - Marker: "Variable concept complete"
 #### 1.4: Statement Concepts (4-5 hours)
 - Abstract Statement parent
 - Concrete: Block, Assignment, IfStatement, WhileLoop, ForLoop, Return, ExpressionStatement
 - Each with appropriate properties and references
 - Marker: "All 7 statement types complete"
 #### 1.5: Expression Concepts (4-5 hours)
 - Abstract Expression parent
 - Binary/Unary operations with operator properties
 - Function call, variable reference, literal types (5 literal types)
 - List literal, index access, member access
 - Total: 13 expression concept types
 - Marker: "All 13 expression types complete"
 #### 1.6: Type Concepts (3-4 hours)
 - Abstract Type parent
 - Concrete: PrimitiveType (with enum), ListType, SetType, MapType, TupleType, ArrayType, OptionalType, CustomType
 - 8 concrete type concepts
 - Ensure ListType.elementType references Type (composable)
 - MapType.keyType and valueType both reference Type
 - TupleType.elementTypes are ordered, multi-valued Type references
 - Marker: "All 8 type concepts complete, nesting tested"
 #### 1.7: Annotation Concepts (2-3 hours)
 - Abstract Annotation parent (optional, for future extensibility)
 - DerefStrategy: strategy enum, derefTime?, derefLocation?, owner?
 - OptimizationLock: lockedBy, lockReason, lockLevel enum, affectedStrategies[], timestamp
 - LangSpecific: language enum, idiomType, rawSyntax, semanticHint?, position enum
 - Marker: "All 3 annotation types complete"
 #### 1.8: Constraints & Validation (2 hours)
 - Create SemAnno.constraints.mps
 - Basic constraints: statements only in statement lists, expressions only in expression contexts, types reference only Type concepts
 - Can-be-parent/can-be-child rules to prevent invalid structures
 - Marker: "Constraints model complete"
 #### 1.9: Test Model Creation (2-3 hours)
 - Create `languages/SemAnno/tests/SimpleExample.mps` demonstrating:
  - 1 Module with name "example"
  - 1 Function "sum" with 2 parameters (items: list[int], factor: float), return type: float
  - Function body with: ForLoop (iterate items), Assignment (accumulate), IfStatement (conditional factor), Return
  - 1 Variable with complex type: optional[map[string, list[int]]]
  - Annotations: DerefStrategy on function, OptimizationLock metadata, LangSpecific on variable
 - Verify model loads without errors
 - Marker: "Test model created and validated"
 **Estimated Duration**: 20-25 hours of focused implementation
 **Quality Gates**:
 - SemAnno language compiles with zero errors
 - Test model loads and renders without errors
 - All concept definitions are complete (no stub concepts)
 ---
 ### Phase 2: Editor Definition
 **Objectives**: Define editors for all concepts in SemAnno.editor.mps so AST is manually creatable and editable in MPS
 **Deliverables**:
 - SemAnno.editor.mps with editors for all 30+ concepts
 - Editors use MPS cell models (constant, property, reference cells) with layout (horizontal, vertical, indent)
 - Syntax highlighting: keywords in blue (def, return, if, for, while), strings in green, numbers in magenta, types in blue
 - No projection switching yet (that comes in Phase 2 of Python/C++ features)
 **Key Subtasks**:
 #### 2.1: Container & Function Editors (2-3 hours)
 - Module editor: "module" keyword, name property, vertical list of variables and functions
 - Function editor: "def" keyword, name, parameters in parens separated by commas, "->", return type, ":", function body with indentation
 - Parameter editor: name, ":", type reference
 - Block editor: indented statements vertically
 #### 2.2: Statement Editors (4-5 hours)
 - Assignment: target expression, "=", value expression
 - IfStatement: "if" keyword (blue), condition, ":", then-branch indented, optional "else:" keyword, else-branch indented
 - WhileLoop: "while" keyword, condition, ":", body indented
 - ForLoop: "for" keyword, iterator name, "in" keyword, iterable expression, ":", body indented
 - Return: "return" keyword, optional return expression
 - ExpressionStatement: expression only
 - Indentation and keyword highlighting applied consistently
 #### 2.3: Expression Editors (3-4 hours)
 - BinaryOperation: left operand, operator property, right operand
 - UnaryOperation: operator, operand
 - FunctionCall: function name, "(", arguments comma-separated, ")"
 - VariableReference: variable name property
 - Literal editors: each shows its value with appropriate syntax highlighting (numbers, strings, keywords)
 - ListLiteral: "[", elements comma-separated, "]"
 - IndexAccess: target, "[", index, "]"
 - MemberAccess: target, ".", member name
 #### 2.4: Type Editors (2-3 hours)
 - PrimitiveType: enum displayed as name (int, float, string, bool)
 - ListType: "[", element type, "]"
 - SetType: "set[", element type, "]"
 - MapType: "map[", key type, ",", value type, "]"
 - TupleType: "tuple[", element types comma-separated, "]"
 - ArrayType: type, "[", size expression, "]"
 - OptionalType: type, "?"
 - CustomType: type name property
 - Type nesting renders naturally: "list[map[string, int]]"
 #### 2.5: Annotation Editors (1-2 hours)
 - DerefStrategy: "@deref(", strategy enum, optional properties in comments/sub-lines, ")"
 - OptimizationLock: "@lock(", lockedBy, ",", lockReason, optional properties, ")"
 - LangSpecific: "@lang_specific(", language, ",", idiomType, ",", rawSyntax, optional properties, ")"
 - Annotations render near the node they modify (before/after depending on position)
 #### 2.6: Editor Testing in MPS (2-3 hours)
 - Rebuild language in MPS
 - Manually create Module in test model
 - Test each concept's editor: create, edit properties, add references
 - Verify indentation, syntax highlighting, navigation work correctly
 - Test nested structures (function → body → statements → expressions → nested types)
 **Estimated Duration**: 15-18 hours
 **Quality Gates**:
 - All editors render without cell errors
 - All properties are editable inline
 - All references are selectable from dropdown/search
 - Indentation and syntax highlighting work as specified
 - Nested structures (functions with statements, statements with expressions, types with types) display correctly
 ---
 ### Phase 3: Integration & Testing
 **Objectives**: Verify AST structure works end-to-end, document for next phase, prepare for Python/C++ projections
 **Deliverables**:
 - Comprehensive test model (SemAnno.tests.mps) with multiple examples
 - Behavior model stubs (SemAnno.behavior.mps) - empty for now, but structure in place
 - Typesystem model stub (SemAnno.typesystem.mps) - empty for now
 - Quick-start guide for creating AST in MPS
 **Key Subtasks**:
 #### 3.1: Extended Test Model (2-3 hours)
 - Add 3-5 example functions covering:
  - Simple function (no loops or conditionals)
  - Function with nested loops and conditionals
  - Function with complex types (nested collections, optional types)
  - Function with annotations (DerefStrategy, OptimizationLock)
  - Function with language-specific idiom annotations
 - Verify all examples load and render without errors
 #### 3.2: Model Serialization Tests (1-2 hours)
 - Save test models to disk (automatic via MPS)
 - Close and reopen MPS
 - Reload test models and verify no corruption
 - Verify all editor state preserved
 #### 3.3: Quality & Correctness Checks (1-2 hours)
 - Run SemAnno language rebuild: zero errors, zero warnings
 - Run MPS consistency checks on test models
 - Verify no dangling references or broken hierarchy
 - Document any edge cases found (e.g., circular type references if possible)
 #### 3.4: Documentation & Quickstart (1-2 hours)
 - Create `languages/SemAnno/QUICKSTART.md` with steps to:
  - Create a new SemAnno model
  - Add a Module and Function
  - Add parameters and statements
  - Create type references
  - Add annotations
 - Include screenshots or ASCII examples of expected editor rendering
 #### 3.5: Prepare for Phase 2 (Python Projection) (1 hour)
 - Note any editor patterns that will need enhancement for projections
 - Document how Python/C++ projection editors will overlay the core AST editors
 - Ensure no blocking issues for projection implementation
 **Estimated Duration**: 7-10 hours
 **Quality Gates**:
 - All test models load, render, and save without errors
 - Serialization round-trips (save→close→reopen) work correctly
 - Zero errors from MPS language rebuild and consistency checks
 - Quickstart documentation is clear and complete
 ---
 ## Phase Completion Criteria
 ### Phase 1 Completion
 - [ ] All 30+ concept definitions in SemAnno.structure.mps complete
 - [ ] SemAnno.constraints.mps defines validation rules
 - [ ] SimpleExample.mps test model created with multi-statement, multi-type function
 - [ ] MPS rebuild produces zero errors, zero warnings
 ### Phase 2 Completion
 - [ ] All concept editors defined in SemAnno.editor.mps
 - [ ] Manual AST creation works in MPS (can create Module → Function → Statements → Expressions)
 - [ ] Syntax highlighting, indentation, and reference selection work correctly
 - [ ] Complex nested types display correctly (e.g., "list[map[string, optional[int]]]")
 ### Phase 3 Completion
 - [ ] Extended test models (3-5 examples) all load without errors
 - [ ] Save/close/reopen cycle works without corruption
 - [ ] Zero errors from language rebuild and consistency checks
 - [ ] Quickstart guide complete and documented
 ---
 ## Data Model
 ### Entities
 **Module** (root AST)
 - Properties: name (string)
 - References: annotations (Annotation*), variables (Variable*), functions (Function*)
 **Function**
 - Properties: name (string)
 - References: parameters (Parameter*), returnType (Type), body (Statement*), annotations (Annotation*)
 **Parameter**
 - Properties: name (string)
 - References: type (Type), defaultValue (Expression?)
 **Variable**
 - Properties: name (string)
 - References: type (Type), initializer (Expression?), annotations (Annotation*)
 **Statement** (abstract)
 - Subtypes: Block, Assignment, IfStatement, WhileLoop, ForLoop, Return, ExpressionStatement
 **Block**
 - References: statements (Statement*)
 **Assignment**
 - References: target (Expression), value (Expression)
 **IfStatement**
 - References: condition (Expression), thenBranch (Statement*), elseBranch (Statement*)
 **WhileLoop**
 - References: condition (Expression), body (Statement*)
 **ForLoop**
 - Properties: iteratorName (string)
 - References: iterable (Expression), body (Statement*)
 **Return**
 - References: value (Expression?)
 **ExpressionStatement**
 - References: expression (Expression)
 **Expression** (abstract)
 - Subtypes: BinaryOperation, UnaryOperation, FunctionCall, VariableReference, Literals, ListLiteral, IndexAccess, MemberAccess
 **BinaryOperation**
 - Properties: operator (string)
 - References: left (Expression), right (Expression)
 **UnaryOperation**
 - Properties: operator (string)
 - References: operand (Expression)
 **FunctionCall**
 - Properties: functionName (string)
 - References: arguments (Expression*)
 **VariableReference**
 - Properties: variableName (string)
 **Literals**
 - IntegerLiteral: value (int)
 - FloatLiteral: value (float)
 - StringLiteral: value (string)
 - BooleanLiteral: value (bool)
 - NullLiteral: (no properties)
 **ListLiteral**
 - References: elements (Expression*)
 **IndexAccess**
 - References: target (Expression), index (Expression)
 **MemberAccess**
 - Properties: memberName (string)
 - References: target (Expression)
 **Type** (abstract)
 - Subtypes: PrimitiveType, ListType, SetType, MapType, TupleType, ArrayType, OptionalType, CustomType
 **PrimitiveType**
 - Properties: kind (enum: int, float, string, bool)
 **ListType**
 - References: elementType (Type)
 **SetType**
 - References: elementType (Type)
 **MapType**
 - References: keyType (Type), valueType (Type)
 **TupleType**
 - References: elementTypes (Type*)
 **ArrayType**
 - References: elementType (Type), size (Expression)
 **OptionalType**
 - References: innerType (Type)
 **CustomType**
 - Properties: typeName (string)
 **Annotation** (abstract)
 - Subtypes: DerefStrategy, OptimizationLock, LangSpecific
 **DerefStrategy**
 - Properties: strategy (enum: imperative, streamed, batched, content_addressed), derefLocation (string?), owner (string?)
 - References: derefTime (Expression?)
 **OptimizationLock**
 - Properties: lockedBy (string), lockReason (string), lockLevel (enum: warning, soft, hard), affectedStrategies (string*), timestamp (string)
 **LangSpecific**
 - Properties: language (enum: python, cpp, rust), idiomType (string), rawSyntax (string), semanticHint (string?), position (enum: before, after, wrapping)
 ---
 ## Risk Mitigation
 | Risk | Likelihood | Impact | Mitigation |
 |------|-----------|--------|-----------|
 | Circular type references (e.g., Type → Type) | Medium | High | Research MPS patterns during Phase 1; use composition, not inheritance |
 | Editor cell model complexity | Medium | Medium | Reference MPS documentation and examples; iterate incrementally |
 | Missing statement/expression types | Low | Medium | Verify against spec requirements before starting; use checklist |
 | Annotation attachment logic issues | Low | Medium | Test annotation creation manually; verify multi-valued references work |
 | Serialization/persistence issues | Low | High | Test save/close/reopen cycle extensively in Phase 3 |
 ---
 ## Timeline Estimate
 **Total Duration**: 42-53 hours (5-7 working days at 8 hours/day, or 2-3 weeks part-time)
 - Phase 0 (Research): Concurrent with Phase 1, not blocking
 - Phase 1 (Structure): 20-25 hours
 - Phase 2 (Editors): 15-18 hours
 - Phase 3 (Testing & Docs): 7-10 hours
 ---
 ## Success Criteria (from Specification)
 1. ✅ **Manual Creation**: Developer can create Module with Function, statements, expressions in MPS with no red errors
 2. ✅ **AST Completeness**: All concepts from Req 1.1-1.6 and 2.1 defined and working
 3. ✅ **Editor Usability**: Selecting, modifying, deleting nodes works in editor (no manual XML editing)
 4. ✅ **Type System**: Complex nested types render correctly
 5. ✅ **Annotations**: DerefStrategy, OptimizationLock, LangSpecific can be created and attached
 6. ✅ **Model Persistence**: Models save and reload without corruption
 7. ✅ **Zero Errors**: Language compiles with zero errors, zero warnings
 8. ✅ **Test Model**: Reference test model with multi-statement functions and annotations exists
 ---
 ## Next Feature Prerequisites
 Once Phase 3 is complete:
 - Python Projection feature can begin (uses core AST + Python-specific editors)
 - C++ Projection feature can begin (uses core AST + C++-specific editors)
 - Behavior model (SemAnno.behavior.mps) can be populated with import logic
 - Typesystem model (SemAnno.typesystem.mps) can define type checking rules
 No blockers expected. Core AST is standalone and self-contained.
--- a/features/001-core-ast-structure/tasks.md
+++ b/features/001-core-ast-structure/tasks.md
@@ -0,0 +1,847 @@
 # Implementation Tasks: Core AST Structure (Feature 001)
 **Feature**: Core AST Structure for SemAnno
 **Phase**: Planning Complete → Ready for Phase 1 Execution
 **Total Tasks**: 26
 **Estimated Duration**: 42-53 hours
 ---
 ## Legend
 - **[PHASE]**: 0 (Research), 1 (Structure), 2 (Editors), 3 (Testing)
 - **[PRIORITY]**: P0 (blocker), P1 (must-have), P2 (should-have), P3 (nice-to-have)
 - **[DURATION]**: Estimated hours
 - **[DEPENDS]**: Task IDs that must complete first
 ---
 ## Phase 0: Research & Exploration
 ### RESEARCH-1: MPS Editor Design Patterns for Projections
 **Priority**: P1 | **Duration**: 2h | **Depends**: None
 **Status**: Pending
 **Objective**: Research how MPS languages support multiple projections (e.g., Python and C++ views on same AST).
 **Tasks**:
 - Read MPS Language Design documentation on projection cells
 - Find 2-3 examples in jetbrains.mps.lang.* languages (e.g., editor, structure)
 - Document patterns: how are editors structured to allow runtime projection switching?
 - Create summary: "Editor Design Patterns" with code examples
 **Definition of Done**:
 - Summary document created
 - At least 2 patterns identified and documented
 - Phase 1 editor design can proceed using these patterns
 ---
 ### RESEARCH-2: Nested Type Composition in MPS
 **Priority**: P1 | **Duration**: 2h | **Depends**: None
 **Status**: Pending
 **Objective**: Verify that deeply nested type references don't cause circular dependencies or serialization issues.
 **Tasks**:
 - Research MPS reference composition patterns
 - Test if Type → Type references can create lists[map[string, int]] without circular deps
 - Document: How to prevent cycles, how to serialize nested structures
 - Prepare example: 3-4 levels of nesting (list[map[string, optional[int]]])
 **Definition of Done**:
 - Patterns documented
 - Circular reference prevention strategy clear
 - Example type hierarchy designed
 ---
 ### RESEARCH-3: Annotation Attachment Patterns
 **Priority**: P1 | **Duration**: 2h | **Depends**: None
 **Status**: Pending
 **Objective**: Understand best practices for attaching multiple independent annotations to a single node.
 **Tasks**:
 - Research multi-valued references in MPS
 - Study how jetbrains.mps.lang.core.structure handles attributes/annotations
 - Verify: Can a Function have [DerefStrategy, OptimizationLock, LangSpecific] simultaneously?
 - Document ordering, optional/required logic, removal behavior
 **Definition of Done**:
 - Annotation design pattern documented
 - Multi-valued reference behavior verified
 - Phase 1 structure design can proceed
 ---
 ## Phase 1: Core Structure Definition
 ### TASK-1-1: Define Root & Container Concepts
 **Priority**: P0 | **Duration**: 2h | **Depends**: None
 **Phase**: 1
 **Status**: Pending
 **Objective**: Define Module concept and establish container pattern.
 **Subtasks**:
 1. Create Module concept in SemAnno.structure.mps
   - name: string property
   - annotations: Annotation* reference (multi-valued)
   - variables: Variable* reference (multi-valued)
   - functions: Function* reference (multi-valued)
 2. Verify MPS recognizes all properties and references
 3. Mark concept as "root" in MPS (if applicable)
 **Definition of Done**:
 - Module concept defined and saved
 - MPS structure editor shows concept without errors
 - All 4 features (name property, 3 references) visible in model tree
 **Review Criteria**:
 - [ ] Concept name, property names match spec
 - [ ] References are multi-valued where appropriate
 - [ ] No red errors in MPS
 ---
 ### TASK-1-2: Define Function & Parameter Concepts
 **Priority**: P0 | **Duration**: 3h | **Depends**: TASK-1-1
 **Phase**: 1
 **Status**: Pending
 **Objective**: Define Function and Parameter concepts with proper signature structure.
 **Subtasks**:
 1. Create Function concept
   - name: string property
   - parameters: Parameter* reference (multi-valued, ordered)
   - returnType: Type reference (single)
   - body: Statement* reference (multi-valued, ordered)
   - annotations: Annotation* reference (multi-valued)
 2. Create Parameter concept
   - name: string property
   - type: Type reference (single)
   - defaultValue: Expression? reference (optional)
 3. Verify Function.parameters and Parameter.type are composable
 4. Test: Create Function, add 2 Parameters with types, set returnType
 **Definition of Done**:
 - Function and Parameter concepts defined
 - Parameter ordering preserved
 - Type references are composable (Type → Type reference works)
 - Can create function with 2 parameters in test model without errors
 ---
 ### TASK-1-3: Define Variable Concept
 **Priority**: P0 | **Duration**: 1h | **Depends**: TASK-1-1
 **Phase**: 1
 **Status**: Pending
 **Objective**: Define Variable concept for module and local variables.
 **Subtasks**:
 1. Create Variable concept
   - name: string property
   - type: Type reference (single)
   - initializer: Expression? reference (optional)
   - annotations: Annotation* reference (multi-valued)
 2. Ensure Variable can be added to Module.variables
 3. Test: Create Module → add Variable with type and initializer
 **Definition of Done**:
 - Variable concept defined
 - Can be added to Module and function scopes
 - No errors in test model
 ---
 ### TASK-1-4: Define All Statement Concepts
 **Priority**: P0 | **Duration**: 5h | **Depends**: TASK-1-2
 **Phase**: 1
 **Status**: Pending
 **Objective**: Define abstract Statement parent and 7 concrete statement types.
 **Subtasks**:
 1. Create abstract Statement concept (parent)
 2. Create concrete statement concepts:
   - **Block**: statements: Statement* (multi-valued)
   - **Assignment**: target: Expression, value: Expression
   - **IfStatement**: condition: Expression, thenBranch: Statement*, elseBranch: Statement? (optional)
   - **WhileLoop**: condition: Expression, body: Statement*
   - **ForLoop**: iteratorName: string property, iterable: Expression, body: Statement*
   - **Return**: value: Expression? (optional)
   - **ExpressionStatement**: expression: Expression
 3. Verify inheritance hierarchy (all inherit from Statement)
 4. Test: Create function → add each statement type to body
 5. Test nesting: Block → Assignment → BinaryOperation expression
 6. Verify statement ordering is preserved
 **Definition of Done**:
 - All 7 concrete statement types defined
 - All inherit from Statement parent
 - Can create function with mixed statement types
 - Nesting works (statements inside blocks, expressions inside statements)
 - No errors in test model
 **Review Criteria**:
 - [ ] All statement types from spec are defined
 - [ ] Abstract Statement parent established
 - [ ] Multi-valued statement references are ordered
 - [ ] Optional references (elseBranch, return value) work correctly
 ---
 ### TASK-1-5: Define All Expression Concepts
 **Priority**: P0 | **Duration**: 5h | **Depends**: TASK-1-4
 **Phase**: 1
 **Status**: Pending
 **Objective**: Define abstract Expression parent and 13 concrete expression types.
 **Subtasks**:
 1. Create abstract Expression concept (parent)
 2. Create concrete expression concepts:
   - **BinaryOperation**: operator: string, left: Expression, right: Expression
   - **UnaryOperation**: operator: string, operand: Expression
   - **FunctionCall**: functionName: string, arguments: Expression* (multi-valued)
   - **VariableReference**: variableName: string
   - **IntegerLiteral**: value: integer
   - **FloatLiteral**: value: float
   - **StringLiteral**: value: string
   - **BooleanLiteral**: value: boolean
   - **NullLiteral**: (no properties)
   - **ListLiteral**: elements: Expression* (multi-valued)
   - **IndexAccess**: target: Expression, index: Expression
   - **MemberAccess**: target: Expression, memberName: string
 3. Verify inheritance hierarchy
 4. Test: Create Assignment with BinaryOperation on right side
 5. Test nesting: FunctionCall → BinaryOperation arguments
 6. Test all literal types can be created
 **Definition of Done**:
 - All 13 concrete expression types defined
 - All inherit from Expression parent
 - Can create complex nested expressions
 - Literals can be created and have correct value types
 - No errors in test model
 **Review Criteria**:
 - [ ] All expression types from spec defined
 - [ ] Abstract Expression parent established
 - [ ] Expression nesting works (expressions contain expressions)
 - [ ] Operator strings work as properties
 - [ ] Argument lists for FunctionCall are ordered
 ---
 ### TASK-1-6: Define All Type Concepts
 **Priority**: P0 | **Duration**: 4h | **Depends**: TASK-1-3
 **Phase**: 1
 **Status**: Pending
 **Objective**: Define abstract Type parent and 8 concrete type types with composition support.
 **Subtasks**:
 1. Create abstract Type concept (parent)
 2. Create concrete type concepts:
   - **PrimitiveType**: kind: enum {int, float, string, bool}
   - **ListType**: elementType: Type (reference)
   - **SetType**: elementType: Type (reference)
   - **MapType**: keyType: Type, valueType: Type
   - **TupleType**: elementTypes: Type* (multi-valued, ordered)
   - **ArrayType**: elementType: Type, size: Expression
   - **OptionalType**: innerType: Type
   - **CustomType**: typeName: string
 3. Verify Type → Type references are composable
 4. Test: Create type "list[map[string, int]]" by composition
 5. Test: Create type "optional[tuple[int, string, bool]]"
 6. Verify no circular reference issues
 7. Test: Create Variable with complex type, render in editor
 **Definition of Done**:
 - All 8 concrete type types defined
 - Type references are fully composable (ListType.elementType → MapType.keyType → PrimitiveType)
 - Can create complex nested types without errors
 - PrimitiveType enum values work correctly
 - No circular reference issues detected
 **Review Criteria**:
 - [ ] All type types from spec defined
 - [ ] Type composition works (type → type → type nesting)
 - [ ] Enum values for PrimitiveType are: int, float, string, bool
 - [ ] ArrayType.size accepts Expression references
 - [ ] TupleType elementTypes preserve order
 ---
 ### TASK-1-7: Define All Annotation Concepts
 **Priority**: P1 | **Duration**: 3h | **Depends**: TASK-1-2, TASK-1-5
 **Phase**: 1
 **Status**: Pending
 **Objective**: Define abstract Annotation parent and 3 concrete annotation types.
 **Subtasks**:
 1. Create abstract Annotation concept (parent, optional for extensibility)
 2. Create concrete annotation concepts:
   - **DerefStrategy**:
     - strategy: enum {imperative, streamed, batched, content_addressed}
     - derefTime: Expression? (optional)
     - derefLocation: string? (optional)
     - owner: string? (optional)
   - **OptimizationLock**:
     - lockedBy: string
     - lockReason: string
     - lockLevel: enum {warning, soft, hard}
     - affectedStrategies: string* (multi-valued)
     - timestamp: string
   - **LangSpecific**:
     - language: enum {python, cpp, rust}
     - idiomType: string
     - rawSyntax: string
     - semanticHint: string? (optional)
     - position: enum {before, after, wrapping}
 3. Verify annotations can be attached to Function.annotations
 4. Test: Create Function with 1 DerefStrategy annotation
 5. Test: Create Variable with 1 LangSpecific annotation
 6. Test: Attach multiple annotations to same node
 **Definition of Done**:
 - All 3 concrete annotation types defined
 - Optional properties work (derefTime, owner, semanticHint)
 - Enum values are correct
 - Can attach annotations to functions and variables
 - Multi-valued affectedStrategies list works
 - No errors in test model
 ---
 ### TASK-1-8: Define Constraints & Validation Rules
 **Priority**: P1 | **Duration**: 2h | **Depends**: TASK-1-4, TASK-1-5, TASK-1-6
 **Phase**: 1
 **Status**: Pending
 **Objective**: Create SemAnno.constraints.mps with basic validation to prevent invalid structures.
 **Subtasks**:
 1. Create SemAnno.constraints.mps model file
 2. Define constraints:
   - Statements can only appear in Statement* contexts (Block.statements, Function.body, etc.)
   - Expressions can only appear in Expression contexts
   - Types can only appear in Type contexts
   - Statements cannot be direct children of Expression
   - Expressions cannot be direct children of Statement (only in ExpressionStatement)
 3. Add can-be-parent/can-be-child rules to prevent invalid structures
 4. Test: Attempt to place a Statement inside an Expression context → should be blocked by constraint
 **Definition of Done**:
 - SemAnno.constraints.mps created
 - Basic structural constraints defined
 - MPS enforces constraint violations
 - Invalid structures are rejected during creation
 ---
 ### TASK-1-9: Create Test Model with Complex AST
 **Priority**: P0 | **Duration**: 3h | **Depends**: TASK-1-7
 **Phase**: 1
 **Status**: Pending
 **Objective**: Create SimpleExample.mps demonstrating all concept types.
 **Subtasks**:
 1. Create new SemAnno model file: `languages/SemAnno/tests/SimpleExample.mps`
 2. Create Module with name "example"
 3. Add 1 Function "sum" with:
   - Parameters: items (list[int]), factor (float)
   - returnType: float
   - Body with 4 statements:
     - ForLoop: iterate items, assign accumulator
     - Assignment: result = accumulator * factor
     - IfStatement: if factor > 0 then return result, else return 0
     - Return: return result
 4. Add 1 Variable: cache (optional[map[string, list[int]]])
 5. Add annotations:
   - DerefStrategy on function (strategy: batched)
   - OptimizationLock on function (locked by "alice", reason "optimization")
   - LangSpecific on variable (language: python, idiomType: "type_hint")
 6. Save and reload model in MPS
 7. Verify no errors, no warnings
 **Definition of Done**:
 - SimpleExample.mps created with all concept types demonstrated
 - Model includes: Module, Function, Parameters, Variable, all Statement types, nested Expressions, nested Types, Annotations
 - Model loads and renders without errors
 - Model serializes and deserializes correctly
 - No red errors in MPS error list
 **Acceptance Criteria**:
 - [ ] Module created with name
 - [ ] Function with 2 parameters, return type, body
 - [ ] Body contains ForLoop, Assignment, IfStatement, Return
 - [ ] Variable with complex nested type
 - [ ] All annotations attached and visible
 ---
 ## Phase 1 Completion Checkpoint
 **Validation Checklist**:
 - [ ] All 30+ concepts defined in SemAnno.structure.mps
 - [ ] SemAnno.constraints.mps created with validation rules
 - [ ] SimpleExample.mps created and loads without errors
 - [ ] MPS language rebuild: zero errors, zero warnings
 - [ ] All concept names match specification
 - [ ] All properties and references match specification
 **Approval**: Ready to proceed to Phase 2 (Editors) only if all checkpoints pass.
 ---
 ## Phase 2: Editor Definition
 ### TASK-2-1: Define Module & Function Editors
 **Priority**: P0 | **Duration**: 3h | **Depends**: TASK-1-2
 **Phase**: 2
 **Status**: Pending
 **Objective**: Create editor definitions for Module, Function, Parameter concepts.
 **Subtasks**:
 1. Open SemAnno.editor.mps (create if needed)
 2. Define Module editor:
   - Render: "module" (blue keyword), space, name (editable property), newline
   - List functions and variables vertically, indented
 3. Define Function editor:
   - Render: "def" (blue keyword), space, name, "(", parameters comma-separated, ")", space, "->", space, return type, ":", newline
   - Body statements indented vertically
 4. Define Parameter editor:
   - Render: name, ":", space, type reference
 5. Test in MPS:
   - Create Module, add Function, add Parameters
   - Verify rendering matches expected layout
   - Verify indentation works
 **Definition of Done**:
 - Module, Function, Parameter editors defined
 - Editors render with proper syntax highlighting (blue keywords)
 - Indentation works for nested structures
 - Parameters display comma-separated
 - Return type reference is selectable
 ---
 ### TASK-2-2: Define All Statement Editors
 **Priority**: P0 | **Duration**: 5h | **Depends**: TASK-2-1
 **Phase**: 2
 **Status**: Pending
 **Objective**: Create editor definitions for all 7 statement types.
 **Subtasks**:
 1. Define Block editor:
   - Render statements vertically, indented
 2. Define Assignment editor:
   - Render: target, space, "=", space, value
 3. Define IfStatement editor:
   - Render: "if" (blue), space, condition, ":", newline
   - Then-branch indented
   - Optional "else:" (blue), newline, else-branch indented
 4. Define WhileLoop editor:
   - Render: "while" (blue), space, condition, ":", newline, body indented
 5. Define ForLoop editor:
   - Render: "for" (blue), space, iteratorName, space, "in" (blue), space, iterable, ":", newline, body indented
 6. Define Return editor:
   - Render: "return" (blue), space, optional value expression
 7. Define ExpressionStatement editor:
   - Render: expression
 8. Test: Create function with mixed statements, verify layout
 **Definition of Done**:
 - All 7 statement editors defined
 - Syntax highlighting applied (keywords in blue)
 - Indentation rendered correctly for nested statements
 - Optional branches (else, return value) display when present, hidden when absent
 ---
 ### TASK-2-3: Define All Expression Editors
 **Priority**: P0 | **Duration**: 4h | **Depends**: TASK-2-2
 **Phase**: 2
 **Status**: Pending
 **Objective**: Create editor definitions for all 13 expression types.
 **Subtasks**:
 1. Define BinaryOperation editor:
   - Render: left, space, operator, space, right
 2. Define UnaryOperation editor:
   - Render: operator, operand
 3. Define FunctionCall editor:
   - Render: functionName, "(", arguments comma-separated, ")"
 4. Define VariableReference editor:
   - Render: variableName
 5. Define Literal editors (5 types):
   - IntegerLiteral: value (number color - magenta)
   - FloatLiteral: value (number color - magenta)
   - StringLiteral: '"', value (string color - green), '"'
   - BooleanLiteral: value (keyword color - blue)
   - NullLiteral: "None" or "null" (keyword color - blue)
 6. Define ListLiteral editor:
   - Render: "[", elements comma-separated, "]"
 7. Define IndexAccess editor:
   - Render: target, "[", index, "]"
 8. Define MemberAccess editor:
   - Render: target, ".", memberName
 9. Test: Create complex nested expressions, verify rendering
 **Definition of Done**:
 - All 13 expression editors defined
 - Syntax coloring applied (keywords blue, strings green, numbers magenta)
 - Operators and operands display correctly
 - Nested expressions render with proper precedence indication
 - Literals render with appropriate colors
 ---
 ### TASK-2-4: Define All Type Editors
 **Priority**: P0 | **Duration**: 3h | **Depends**: TASK-2-3
 **Phase**: 2
 **Status**: Pending
 **Objective**: Create editor definitions for all 8 type types.
 **Subtasks**:
 1. Define PrimitiveType editor:
   - Render: kind enum value (int, float, string, bool)
 2. Define ListType editor:
   - Render: "list" (blue), "[", elementType, "]"
 3. Define SetType editor:
   - Render: "set" (blue), "[", elementType, "]"
 4. Define MapType editor:
   - Render: "map" (blue), "[", keyType, ",", space, valueType, "]"
 5. Define TupleType editor:
   - Render: "tuple" (blue), "[", elementTypes comma-separated, "]"
 6. Define ArrayType editor:
   - Render: elementType, "[", size expression, "]"
 7. Define OptionalType editor:
   - Render: innerType, "?"
 8. Define CustomType editor:
   - Render: typeName
 9. Test: Create complex nested types (list[map[string, optional[int]]]), verify rendering
 **Definition of Done**:
 - All 8 type editors defined
 - Type keywords rendered in blue
 - Nested types render naturally (list[map[...]])
 - Type references are selectable
 - Composition works (can select complex types)
 ---
 ### TASK-2-5: Define Annotation Editors
 **Priority**: P1 | **Duration**: 2h | **Depends**: TASK-2-4
 **Phase**: 2
 **Status**: Pending
 **Objective**: Create editor definitions for all 3 annotation types.
 **Subtasks**:
 1. Define DerefStrategy editor:
   - Render: "@deref" (gray), "(", strategy enum, optional properties as sub-lines, ")"
 2. Define OptimizationLock editor:
   - Render: "@lock" (gray), "(", properties comma-separated, ")"
   - Include: lockedBy, lockReason, lockLevel, affectedStrategies, timestamp
 3. Define LangSpecific editor:
   - Render: "@lang_specific" (gray), "(", language, ",", idiomType, ",", rawSyntax, optional properties, ")"
 4. Test: Create function with each annotation type, verify rendering near function
 **Definition of Done**:
 - All 3 annotation editors defined
 - Annotation keywords in gray/italic (distinguishes from code)
 - Enum values display correctly
 - Optional properties hidden when absent
 - Annotations render attached to parent node
 ---
 ### TASK-2-6: Editor Testing & Refinement in MPS
 **Priority**: P0 | **Duration**: 3h | **Depends**: TASK-2-5
 **Phase**: 2
 **Status**: Pending
 **Objective**: Rebuild language in MPS, test editors interactively, fix issues.
 **Subtasks**:
 1. Rebuild SemAnno language in MPS
   - Check for cell model errors
   - Verify no "missing editor" errors
 2. Open SimpleExample.mps and manually verify:
   - Module renders correctly (name, lists functions/variables)
   - Function renders with "def", name, parameters, return type, body
   - All statements render with proper indentation and syntax highlighting
   - All expressions render correctly (operators, operands, nesting)
   - All types render correctly (composition, nesting)
   - Annotations render near their nodes
 3. Test interactive editing:
   - Create new Module, add Function by adding statements
   - Edit statement properties (if condition, loop iterator, etc.)
   - Edit expression operators and operands
   - Change types by selecting from dropdown
   - Navigate between references (e.g., click parameter type → jump to Variable definition)
 4. Fix any rendering issues:
   - Indentation not working → adjust cell layout
   - Colors not applying → verify color codes
   - References not selectable → verify reference cell type
 5. Document any workarounds or editor-specific behavior
 **Definition of Done**:
 - Language rebuilds with zero errors, zero warnings
 - All editors render without errors
 - Interactive editing works (properties editable, references selectable)
 - All syntax highlighting and indentation work correctly
 - SimpleExample.mps renders as designed
 **Acceptance Criteria**:
 - [ ] Language rebuild: zero errors
 - [ ] All concepts have working editors
 - [ ] Indentation works for nested structures
 - [ ] Syntax highlighting works (blue keywords, green strings, magenta numbers)
 - [ ] Properties can be edited inline
 - [ ] References are selectable
 ---
 ## Phase 2 Completion Checkpoint
 **Validation Checklist**:
 - [ ] All concept editors defined in SemAnno.editor.mps
 - [ ] MPS language rebuild: zero errors, zero warnings
 - [ ] SimpleExample.mps renders without editor errors
 - [ ] Manual AST creation works (can add statements, expressions, types)
 - [ ] Syntax highlighting and indentation correct
 - [ ] All references are selectable and navigable
 **Approval**: Ready to proceed to Phase 3 (Testing) only if all checkpoints pass.
 ---
 ## Phase 3: Integration & Testing
 ### TASK-3-1: Create Extended Test Models
 **Priority**: P1 | **Duration**: 3h | **Depends**: TASK-2-6
 **Phase**: 3
 **Status**: Pending
 **Objective**: Create 3-5 test models demonstrating various AST structures.
 **Subtasks**:
 1. Create `SimpleFunction.mps`: Basic function (no loops, minimal types)
   - Function: greet(name: string) -> string
   - Body: Return StringLiteral
 2. Create `ComplexFunction.mps`: Nested control flow
   - Function with ForLoop → IfStatement → BinaryOperation
   - Demonstrates statement nesting
 3. Create `NestedTypes.mps`: Complex type structures
   - Variables with types: list[map[string, optional[int]]], tuple[int, string, bool]
   - Demonstrates type composition
 4. Create `WithAnnotations.mps`: Full example with annotations
   - Function with DerefStrategy, OptimizationLock, LangSpecific
   - Variables with annotations
   - Demonstrates annotation attachment
 5. Save all models and verify load without errors
 **Definition of Done**:
 - 5 test models created
 - Each demonstrates different AST features
 - All load and render without errors
 - Each model documents its purpose in comments (if MPS supports)
 ---
 ### TASK-3-2: Test Model Serialization Round-Trips
 **Priority**: P0 | **Duration**: 2h | **Depends**: TASK-3-1
 **Phase**: 3
 **Status**: Pending
 **Objective**: Verify models persist correctly and don't corrupt on save/reload.
 **Subtasks**:
 1. For each test model:
   - Save model to disk (automatic via MPS)
   - Close MPS completely
   - Reopen MPS and reload project
   - Reload each test model
   - Verify no errors, warnings, or corrupted nodes
   - Verify content matches what was saved
 2. Test model properties:
   - All string properties preserved correctly
   - All enum values preserved
   - All references intact (no broken links)
   - Statement and type ordering preserved
 3. Document any serialization quirks or issues found
 **Definition of Done**:
 - All test models round-trip successfully (save → close → reopen → load)
 - No corruption or data loss detected
 - Content and structure verified after reload
 ---
 ### TASK-3-3: Quality & Correctness Verification
 **Priority**: P0 | **Duration**: 2h | **Depends**: TASK-3-2
 **Phase**: 3
 **Status**: Pending
 **Objective**: Run comprehensive checks to ensure language quality.
 **Subtasks**:
 1. Rebuild SemAnno language in MPS:
   - Check for zero errors, zero warnings
   - Verify all concept definitions are complete
   - Verify all editor definitions are complete
 2. Run MPS consistency checks on all test models:
   - Check for dangling references
   - Check for orphaned nodes
   - Verify hierarchy is valid
 3. Test constraint enforcement:
   - Attempt to create invalid structure (e.g., Statement as Expression)
   - Verify constraint blocks it
 4. Document findings: Are there any edge cases or limitations?
 **Definition of Done**:
 - Language rebuild: zero errors, zero warnings
 - All test models pass consistency checks
 - Constraints work as designed
 - No unresolved issues
 ---
 ### TASK-3-4: Write Quickstart Guide
 **Priority**: P1 | **Duration**: 2h | **Depends**: TASK-3-3
 **Phase**: 3
 **Status**: Pending
 **Objective**: Document how to create AST in SemAnno for future users.
 **Deliverable**: `languages/SemAnno/QUICKSTART.md`
 **Contents**:
 1. Overview: What is SemAnno, what can you do with it
 2. Step-by-step guide: Create first Module
   - Create new model file
   - Add Module concept
   - Set module name
 3. Create first Function
   - Add Function to Module.functions
   - Set function name, return type
   - Add parameters
 4. Create statements and expressions
   - Add statements to function body
   - Show examples: ForLoop, Assignment, Return
   - Show expression examples: BinaryOperation, FunctionCall
 5. Work with types
   - Create primitive type
   - Create nested type (list[map[...]])
   - Show how types compose
 6. Add annotations
   - Attach DerefStrategy to function
   - Attach OptimizationLock to variable
 7. Tips & tricks
   - How to navigate references
   - How to edit properties
   - Keyboard shortcuts (if any)
 8. Examples section
   - Link to test models (SimpleFunction.mps, ComplexFunction.mps, etc.)
   - Show rendered output for each
 **Definition of Done**:
 - QUICKSTART.md created
 - All major workflows documented
 - Step-by-step instructions clear
 - Examples provided with screenshots or ASCII renderings
 - No unexplained concepts
 ---
 ### TASK-3-5: Prepare for Phase 2 Features (Python/C++ Projections)
 **Priority**: P2 | **Duration**: 1h | **Depends**: TASK-3-4
 **Phase**: 3
 **Status**: Pending
 **Objective**: Document how Phase 2 features will extend core AST.
 **Deliverable**: `languages/SemAnno/NEXT_PHASES.md`
 **Contents**:
 1. Overview: What comes next (Python Projection, C++ Projection, Tree-sitter Import)
 2. Python Projection (Week 3-4):
   - Will extend editors to show Python-syntax rendering
   - Will add Python text generator
   - Will not change core AST structure
   - Dependencies: This feature (core AST structure)
 3. C++ Projection (Week 5-6):
   - Will extend editors to show C++-syntax rendering
   - Will add C++ text generator
   - Will add deref strategy translation
   - Dependencies: This feature + Python Projection
 4. Tree-sitter Import (Week 7-8):
   - Will add behavior model (SemAnno.behavior.mps) with import logic
   - Will integrate tree-sitter-python and tree-sitter-cpp
   - Will parse source files into AST
   - Dependencies: This feature + both projections
 5. Warning System (Week 9-10):
   - Will add OptimizationLock enforcement logic
   - Will add warning UI to editors
   - Will add provenance tracking
   - Dependencies: All previous features
 6. Blockers or constraints for next phases (if any)
 **Definition of Done**:
 - NEXT_PHASES.md created
 - Clear roadmap for future features
 - Dependencies documented
 - No surprises in Phase 2+ implementation
 ---
 ## Phase 3 Completion Checkpoint
 **Validation Checklist**:
 - [ ] 5 test models created (SimpleFunction, ComplexFunction, NestedTypes, WithAnnotations, etc.)
 - [ ] All models load and render without errors
 - [ ] Serialization round-trips work (save → close → reopen)
 - [ ] Language rebuild: zero errors, zero warnings
 - [ ] Consistency checks pass
 - [ ] Constraints work as designed
 - [ ] Quickstart guide complete and clear
 - [ ] NEXT_PHASES.md prepared
 **Approval**: Feature 001 (Core AST Structure) is COMPLETE and ready for merge to parent branch.
 ---
 ## Summary
 **Total Tasks**: 26
 **Phases**: 3 (Research, Implementation, Testing)
 **Estimated Duration**: 42-53 hours
 **Status**: Ready for execution
 **Next Step**: Begin Phase 1 (TASK-1-1) with core structure definition.
 ---
 ## Approval Signature
 Once all tasks are complete and checkpoints pass:
 ```
 Feature: 001 - Core AST Structure
 Status: ✅ COMPLETE
 Merged to: sprint-1-ast-redesign (parent branch)
 Timestamp: [Completion date]
 ```