whetstone_DSL/docs/SPRINT_1_PROGRESS.md

Sprint 1 Progress Report

Session: 2025-02-05 — TargetLanguage Enum Fix + Status Assessment

What Was Fixed This Session

Bug: Module TextGen generated both Python AND C++ output regardless of the targetLanguage enum selection.

Root Causes (layered):

1. targetLanguage property was typed as string instead of TargetLanguage enum — Fixed in SemAnno.structure.mps
2. TextGen used NotEqualsExpression (!=) for string comparison — In Java, != does reference comparison, not value comparison, so both conditions were always true
3. After fixing to .equals(), MPS generates getEnum() for enum-typed properties, returning an enum object, not a String — "cpp".equals(enumObject) always returns false

Final Fix: Used MPS's EnumMember_IsOperation (is operation) from jetbrains.mps.lang.smodel:

Python block condition: !(node.targetLanguage.is(cpp))
C++ block condition:    !(node.targetLanguage.is(python))

This generates correct Java: SEnumOperations.isMember(SPropertyOperations.getEnum(...), memberID)

Files Changed:

• languages/SemAnno/models/SemAnno.structure.mps — targetLanguage property type: string → TargetLanguage enum
• languages/SemAnno/models/SemAnno.textGen.mps — Module TextGen conditions use is operation with enum member references
• languages/SemAnno/SemAnno.mpl — Added JDK dependency (may still be needed for future .equals() calls)
• languages/SemAnno/source_gen/SemAnno/textGen/Module_TextGen.java — Regenerated by MPS with correct SEnumOperations.isMember() calls

Key MPS Lessons Learned

• Enum property comparison: Use the is operation from jetbrains.mps.lang.smodel, NOT == or .equals() on strings
• SPropertyAccess on enum properties generates getEnum() (returns enum literal), not getString() (returns String)
• EnumMember_IsOperation (concept 21noJN) with EnumMemberReference (concept 21nZrQ) is the correct pattern
• JDK module dependency must be in the <dependencies> section of .mpl, not just <dependencyVersions>
• InstanceMethodCallOperation concept ID is 1202948039474, index liA8E (NOT 1204053956946 which is IMethodCall abstract interface)

Known Remaining Errors

• Return_TextGen: "The reference value (link) is out of search scope" — Pre-existing, needs investigation
• Some unused registry entries from debugging may remain in textGen.mps — Clean up on next session

---

Sprint 1 Status Assessment

Implementation Order from Requirements:

1. Week 1-2: Core AST Structure — Phase: MOSTLY DONE
2. Week 3-4: Python Projection & Generator — Phase: PARTIAL
3. Week 5-6: C++ Projection & Generator — Phase: PARTIAL
4. Week 7-8: Tree-sitter Import — Phase: NOT STARTED
5. Week 9-10: Warning System & Annotations — Phase: NOT STARTED

---

Detailed Status by Area

1. Structure (SemAnno.structure.mps) — MOSTLY DONE

Fully Defined Concepts:

• Module (with targetLanguage enum: python/cpp/both)
• Function (name, parameters, body, returnType, annotations)
• Variable (name, type, initializer, annotations)
• Parameter (name, type)
• PrimitiveType (kind)
• Assignment, Return, ExpressionStatement, Block
• IfStatement, WhileLoop, ForLoop
• BinaryOperation, UnaryOperation
• FunctionCall, VariableReference
• All Literals: IntegerLiteral, FloatLiteral, StringLiteral, BooleanLiteral, NullLiteral, ListLiteral
• IndexAccess, MemberAccess
• Type nodes: ListType, SetType, MapType, TupleType, ArrayType, OptionalType, CustomType
• Annotations: Memory strategy (initially DerefStrategy, canonical: DeallocateAnnotation/LifetimeAnnotation/ReclaimAnnotation/OwnerAnnotation/AllocateAnnotation), OptimizationLock, LangSpecific
• TargetLanguage enum (python, cpp, both)

Status: All TR-1 core AST nodes are defined. Annotation structures (TR-2, TR-3, TR-4) exist but may need property refinement.

2. TextGen — PARTIAL (Core works, most nodes are stubs)

WORKING (read node properties, generate real output):

• Module_TextGen — Conditional Python/C++ generation with enum check ✅
• Function_TextGen — Generates def name(params): with body (Python style only)
• Variable_TextGen — Generates name: type
• Parameter_TextGen — Generates parameter name
• PrimitiveType_TextGen — Generates type kind
• Assignment_TextGen — Generates name = value
• Return_TextGen — Generates return value (has reference error)
• BinaryOperation_TextGen — Generates left op right
• Block_TextGen — Generates indented statements
• ExpressionStatement_TextGen — Generates expression
• VariableReference_TextGen — Generates variable name

STUBS (output placeholder text, not reading node data):

• IntegerLiteral_TextGen — outputs <int> instead of actual value
• FloatLiteral_TextGen — outputs [value]
• StringLiteral_TextGen — outputs "[value]"
• BooleanLiteral_TextGen — outputs [value]
• NullLiteral_TextGen — outputs null (may be correct)
• ListLiteral_TextGen — outputs [elements]
• IfStatement_TextGen — outputs placeholder text
• WhileLoop_TextGen — outputs placeholder text
• ForLoop_TextGen — outputs placeholder text
• FunctionCall_TextGen — outputs [functionName]([arguments])
• UnaryOperation_TextGen — outputs [operator][operand]
• IndexAccess_TextGen — outputs [target][[index]]
• MemberAccess_TextGen — outputs [target].[memberName]
• All Type TextGens (List, Set, Map, Tuple, Array, Optional, Custom) — placeholders
• DerefStrategy_TextGen — outputs @deref([strategy]) (to be updated to canonical annotation names like @Reclaim(Tracing), @Deallocate(Explicit), etc.)
• OptimizationLock_TextGen — outputs @lock([lockedBy], [lockReason])
• LangSpecific_TextGen — outputs @lang([language]:[idiomType])

MISSING:

• C++ mode generation for Function, Variable, etc. (only Module switches; child nodes always generate Python-style)
• Proper literal value output
• Control flow statement generation (if/while/for)

3. Editor (SemAnno.editor.mps) — EXISTS, NEEDS ASSESSMENT

• Editor definitions exist (~20KB)
• Likely has basic cell layouts for concepts
• Dual Python/C++ projection switching: UNKNOWN (needs investigation)

4. Behavior (SemAnno.behavior.mps) — MINIMAL

• File exists but likely minimal

5. Constraints (SemAnno.constraints.mps) — MINIMAL

• File exists but likely minimal

6. TypeSystem (SemAnno.typesystem.mps) — MINIMAL

• File exists but likely minimal

7. Tree-sitter Integration — NOT STARTED

• No tree-sitter files found in project

8. Warning System — NOT STARTED

• OptimizationLock concept exists in structure
• No warning UI or behavior implementation

9. Test Models — MINIMAL

• Phase1Test.mps — Calculator example (Module with 2 functions, variables, assignments, returns, binary ops)
• No other test models found

---

Recommended Next Steps (Priority Order)

Phase A: Fix Remaining TextGen Stubs (High Priority)

Make all TextGen implementations read actual node data instead of outputting placeholder text.

1. Fix literal TextGens — IntegerLiteral, FloatLiteral, StringLiteral, BooleanLiteral should read their value property
2. Fix control flow TextGens — IfStatement, WhileLoop, ForLoop should read conditions and bodies
3. Fix FunctionCall TextGen — Read function name and arguments
4. Fix UnaryOperation TextGen — Read operator and operand
5. Fix IndexAccess, MemberAccess TextGens — Read targets and members
6. Fix type TextGens — Read inner types for composite types
7. Fix annotation TextGens — Read annotation properties

Phase B: Dual Language Generation (High Priority)

Currently only Module switches between Python/C++. Child nodes need language-aware generation:

1. Pass targetLanguage context to child TextGens (or have children read parent Module's targetLanguage)
2. Function_TextGen — Python: def name(params): / C++: returnType name(params) {}
3. Variable_TextGen — Python: name: type / C++: type name;
4. Statement TextGens — Python uses : and indentation / C++ uses {} and ;

Phase C: Fix Return_TextGen Error

• Investigate "The reference value (link) is out of search scope" error
• May be a link ID mismatch in the structure

Phase D: Expand Phase1Test

• Add more test cases: control flow, function calls, literals, type variations
• Test both Python and C++ output for each case

Phase E: Editor Dual Projection (Medium Priority)

• Investigate current editor state
• Add projection switching (Python view vs C++ view)

Phase F: Tree-sitter Import (Lower Priority for Sprint 1)

• External tooling integration
• Can be deferred if core generation works

Phase G: Warning System (Lower Priority)

• OptimizationLock behavior
• Warning UI in editor