bill/whetstone_DSL
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Phase1Example working on choosing from enums actually done

Browse Source
This commit is contained in:
Bill
2026-02-06 01:40:37 -07:00
parent 9b420ba658
commit e763c2c32b
6 changed files with 219 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.claude/settings.local.json
View File

@@ -4,7 +4,11 @@
"Bash(ls:*)", "Bash(ls:*)",
"Bash(git add:*)", "Bash(git add:*)",
"Bash(git commit -m \"$\\(cat <<''EOF''\nAdd textgen definitions for all Phase 1 AST concepts\n\n- Added ConceptTextGenDeclaration nodes for all 38 Phase 1 concepts\n- Includes textgen rules for core concepts \\(Module, Function, Parameter, Variable\\)\n- Textgen rules for statements \\(Block, Assignment, If, Loops, Return, etc.\\)\n- Textgen rules for expressions \\(BinaryOp, UnaryOp, FunctionCall, Literals, etc.\\)\n- Textgen rules for types \\(Primitive, List, Set, Map, Tuple, Array, Optional, Custom\\)\n- Textgen rules for annotations \\(Annotation, DerefStrategy, OptimizationLock, LangSpecific\\)\n\nCo-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>\nEOF\n\\)\")", "Bash(git commit -m \"$\\(cat <<''EOF''\nAdd textgen definitions for all Phase 1 AST concepts\n\n- Added ConceptTextGenDeclaration nodes for all 38 Phase 1 concepts\n- Includes textgen rules for core concepts \\(Module, Function, Parameter, Variable\\)\n- Textgen rules for statements \\(Block, Assignment, If, Loops, Return, etc.\\)\n- Textgen rules for expressions \\(BinaryOp, UnaryOp, FunctionCall, Literals, etc.\\)\n- Textgen rules for types \\(Primitive, List, Set, Map, Tuple, Array, Optional, Custom\\)\n- Textgen rules for annotations \\(Annotation, DerefStrategy, OptimizationLock, LangSpecific\\)\n\nCo-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>\nEOF\n\\)\")",
"Bash(git commit:*)" "Bash(git commit:*)",
"WebFetch(domain:raw.githubusercontent.com)",
"WebSearch",
"WebFetch(domain:mps-support.jetbrains.com)",
"WebFetch(domain:www.jetbrains.com)"
] ]
} }
} }

2
Phase1Example.sandbox/models/Phase1Test.mps
View File

@@ -51,7 +51,7 @@
</registry> </registry>
<node concept="3rCcZv" id="Calc_M001"> <node concept="3rCcZv" id="Calc_M001">
<property role="TrG5h" value="Calculator" /> <property role="TrG5h" value="Calculator" />
<property role="2EAK5_" value="c" /> <property role="2EAK5_" value="6LZhwXW98Rc/cpp" />
<node concept="2EAK29" id="Calc_V001" role="2EAK5V"> <node concept="2EAK29" id="Calc_V001" role="2EAK5V">
<property role="TrG5h" value="PI" /> <property role="TrG5h" value="PI" />
<node concept="3rC1Zv" id="Calc_VT001" role="2EAK2b"> <node concept="3rC1Zv" id="Calc_VT001" role="2EAK2b">

179
docs/SPRINT_1_PROGRESS.md Normal file
View File

@@ -0,0 +1,179 @@
# Sprint 1 Progress Report
## Session: 2025-02-05 — TargetLanguage Enum Fix + Status Assessment
### What Was Fixed This Session
**Bug:** Module TextGen generated both Python AND C++ output regardless of the `targetLanguage` enum selection.
**Root Causes (layered):**
1. `targetLanguage` property was typed as `string` instead of `TargetLanguage` enum — Fixed in `SemAnno.structure.mps`
2. TextGen used `NotEqualsExpression` (`!=`) for string comparison — In Java, `!=` does reference comparison, not value comparison, so both conditions were always `true`
3. After fixing to `.equals()`, MPS generates `getEnum()` for enum-typed properties, returning an enum object, not a String — `"cpp".equals(enumObject)` always returns `false`
**Final Fix:** Used MPS's `EnumMember_IsOperation` (`is` operation) from `jetbrains.mps.lang.smodel`:
```
Python block condition: !(node.targetLanguage.is(cpp))
C++ block condition: !(node.targetLanguage.is(python))
```
This generates correct Java: `SEnumOperations.isMember(SPropertyOperations.getEnum(...), memberID)`
**Files Changed:**
- `languages/SemAnno/models/SemAnno.structure.mps` — `targetLanguage` property type: `string` → `TargetLanguage` enum
- `languages/SemAnno/models/SemAnno.textGen.mps` — Module TextGen conditions use `is` operation with enum member references
- `languages/SemAnno/SemAnno.mpl` — Added JDK dependency (may still be needed for future `.equals()` calls)
- `languages/SemAnno/source_gen/SemAnno/textGen/Module_TextGen.java` — Regenerated by MPS with correct `SEnumOperations.isMember()` calls
### Key MPS Lessons Learned
- **Enum property comparison:** Use the `is` operation from `jetbrains.mps.lang.smodel`, NOT `==` or `.equals()` on strings
- **`SPropertyAccess` on enum properties** generates `getEnum()` (returns enum literal), not `getString()` (returns String)
- **`EnumMember_IsOperation`** (concept `21noJN`) with **`EnumMemberReference`** (concept `21nZrQ`) is the correct pattern
- **JDK module dependency** must be in the `<dependencies>` section of `.mpl`, not just `<dependencyVersions>`
- **InstanceMethodCallOperation** concept ID is `1202948039474`, index `liA8E` (NOT `1204053956946` which is `IMethodCall` abstract interface)
### Known Remaining Errors
- `Return_TextGen`: "The reference value (link) is out of search scope" — Pre-existing, needs investigation
- Some unused registry entries from debugging may remain in textGen.mps — Clean up on next session
---
## Sprint 1 Status Assessment
### Implementation Order from Requirements:
1. **Week 1-2: Core AST Structure** — Phase: MOSTLY DONE
2. **Week 3-4: Python Projection & Generator** — Phase: PARTIAL
3. **Week 5-6: C++ Projection & Generator** — Phase: PARTIAL
4. **Week 7-8: Tree-sitter Import** — Phase: NOT STARTED
5. **Week 9-10: Warning System & Annotations** — Phase: NOT STARTED
---
### Detailed Status by Area
#### 1. Structure (SemAnno.structure.mps) — MOSTLY DONE
**Fully Defined Concepts:**
- Module (with targetLanguage enum: python/cpp/both)
- Function (name, parameters, body, returnType, annotations)
- Variable (name, type, initializer, annotations)
- Parameter (name, type)
- PrimitiveType (kind)
- Assignment, Return, ExpressionStatement, Block
- IfStatement, WhileLoop, ForLoop
- BinaryOperation, UnaryOperation
- FunctionCall, VariableReference
- All Literals: IntegerLiteral, FloatLiteral, StringLiteral, BooleanLiteral, NullLiteral, ListLiteral
- IndexAccess, MemberAccess
- Type nodes: ListType, SetType, MapType, TupleType, ArrayType, OptionalType, CustomType
- Annotations: DerefStrategy, OptimizationLock, LangSpecific
- TargetLanguage enum (python, cpp, both)
**Status:** All TR-1 core AST nodes are defined. Annotation structures (TR-2, TR-3, TR-4) exist but may need property refinement.
#### 2. TextGen — PARTIAL (Core works, most nodes are stubs)
**WORKING (read node properties, generate real output):**
- `Module_TextGen` — Conditional Python/C++ generation with enum check ✅
- `Function_TextGen` — Generates `def name(params):` with body (Python style only)
- `Variable_TextGen` — Generates `name: type`
- `Parameter_TextGen` — Generates parameter name
- `PrimitiveType_TextGen` — Generates type kind
- `Assignment_TextGen` — Generates `name = value`
- `Return_TextGen` — Generates `return value` (has reference error)
- `BinaryOperation_TextGen` — Generates `left op right`
- `Block_TextGen` — Generates indented statements
- `ExpressionStatement_TextGen` — Generates expression
- `VariableReference_TextGen` — Generates variable name
**STUBS (output placeholder text, not reading node data):**
- `IntegerLiteral_TextGen` — outputs `<int>` instead of actual value
- `FloatLiteral_TextGen` — outputs `[value]`
- `StringLiteral_TextGen` — outputs `"[value]"`
- `BooleanLiteral_TextGen` — outputs `[value]`
- `NullLiteral_TextGen` — outputs `null` (may be correct)
- `ListLiteral_TextGen` — outputs `[elements]`
- `IfStatement_TextGen` — outputs placeholder text
- `WhileLoop_TextGen` — outputs placeholder text
- `ForLoop_TextGen` — outputs placeholder text
- `FunctionCall_TextGen` — outputs `[functionName]([arguments])`
- `UnaryOperation_TextGen` — outputs `[operator][operand]`
- `IndexAccess_TextGen` — outputs `[target][[index]]`
- `MemberAccess_TextGen` — outputs `[target].[memberName]`
- All Type TextGens (List, Set, Map, Tuple, Array, Optional, Custom) — placeholders
- `DerefStrategy_TextGen` — outputs `@deref([strategy])`
- `OptimizationLock_TextGen` — outputs `@lock([lockedBy], [lockReason])`
- `LangSpecific_TextGen` — outputs `@lang([language]:[idiomType])`
**MISSING:**
- C++ mode generation for Function, Variable, etc. (only Module switches; child nodes always generate Python-style)
- Proper literal value output
- Control flow statement generation (if/while/for)
#### 3. Editor (SemAnno.editor.mps) — EXISTS, NEEDS ASSESSMENT
- Editor definitions exist (~20KB)
- Likely has basic cell layouts for concepts
- Dual Python/C++ projection switching: UNKNOWN (needs investigation)
#### 4. Behavior (SemAnno.behavior.mps) — MINIMAL
- File exists but likely minimal
#### 5. Constraints (SemAnno.constraints.mps) — MINIMAL
- File exists but likely minimal
#### 6. TypeSystem (SemAnno.typesystem.mps) — MINIMAL
- File exists but likely minimal
#### 7. Tree-sitter Integration — NOT STARTED
- No tree-sitter files found in project
#### 8. Warning System — NOT STARTED
- OptimizationLock concept exists in structure
- No warning UI or behavior implementation
#### 9. Test Models — MINIMAL
- `Phase1Test.mps` — Calculator example (Module with 2 functions, variables, assignments, returns, binary ops)
- No other test models found
---
## Recommended Next Steps (Priority Order)
### Phase A: Fix Remaining TextGen Stubs (High Priority)
Make all TextGen implementations read actual node data instead of outputting placeholder text.
1. **Fix literal TextGens** — IntegerLiteral, FloatLiteral, StringLiteral, BooleanLiteral should read their `value` property
2. **Fix control flow TextGens** — IfStatement, WhileLoop, ForLoop should read conditions and bodies
3. **Fix FunctionCall TextGen** — Read function name and arguments
4. **Fix UnaryOperation TextGen** — Read operator and operand
5. **Fix IndexAccess, MemberAccess TextGens** — Read targets and members
6. **Fix type TextGens** — Read inner types for composite types
7. **Fix annotation TextGens** — Read annotation properties
### Phase B: Dual Language Generation (High Priority)
Currently only Module switches between Python/C++. Child nodes need language-aware generation:
1. **Pass targetLanguage context** to child TextGens (or have children read parent Module's targetLanguage)
2. **Function_TextGen** — Python: `def name(params):` / C++: `returnType name(params) {}`
3. **Variable_TextGen** — Python: `name: type` / C++: `type name;`
4. **Statement TextGens** — Python uses `:` and indentation / C++ uses `{}` and `;`
### Phase C: Fix Return_TextGen Error
- Investigate "The reference value (link) is out of search scope" error
- May be a link ID mismatch in the structure
### Phase D: Expand Phase1Test
- Add more test cases: control flow, function calls, literals, type variations
- Test both Python and C++ output for each case
### Phase E: Editor Dual Projection (Medium Priority)
- Investigate current editor state
- Add projection switching (Python view vs C++ view)
### Phase F: Tree-sitter Import (Lower Priority for Sprint 1)
- External tooling integration
- Can be deferred if core generation works
### Phase G: Warning System (Lower Priority)
- OptimizationLock behavior
- Warning UI in editor

1
languages/SemAnno/SemAnno.mpl
View File

@@ -60,6 +60,7 @@
<dependency reexport="false">f3061a53-9226-4cc5-a443-f952ceaf5816(jetbrains.mps.baseLanguage)</dependency> <dependency reexport="false">f3061a53-9226-4cc5-a443-f952ceaf5816(jetbrains.mps.baseLanguage)</dependency>
<dependency reexport="false">b83431fe-5c8f-40bc-8a36-65e25f4dd253(jetbrains.mps.lang.textGen)</dependency> <dependency reexport="false">b83431fe-5c8f-40bc-8a36-65e25f4dd253(jetbrains.mps.lang.textGen)</dependency>
<dependency reexport="false">ceab5195-25ea-4f22-9b92-103b95ca8c0c(jetbrains.mps.lang.core)</dependency> <dependency reexport="false">ceab5195-25ea-4f22-9b92-103b95ca8c0c(jetbrains.mps.lang.core)</dependency>
<dependency reexport="false">6354ebe7-c22a-4a0f-ac54-50b52ab9b065(JDK)</dependency>
</dependencies> </dependencies>
<languageVersions> <languageVersions>
<language slang="l:f3061a53-9226-4cc5-a443-f952ceaf5816:jetbrains.mps.baseLanguage" version="12" /> <language slang="l:f3061a53-9226-4cc5-a443-f952ceaf5816:jetbrains.mps.baseLanguage" version="12" />

2
languages/SemAnno/models/SemAnno.structure.mps
View File

@@ -172,7 +172,7 @@
<node concept="1TJgyi" id="7kypvuIwECI" role="1TKVEl"> <node concept="1TJgyi" id="7kypvuIwECI" role="1TKVEl">
<property role="IQ2nx" value="8982541288447632592" /> <property role="IQ2nx" value="8982541288447632592" />
<property role="TrG5h" value="targetLanguage" /> <property role="TrG5h" value="targetLanguage" />
<ref role="AX2Wp" to="tpck:fKAOsGN" resolve="string" /> <ref role="AX2Wp" node="6LZhwXW979c" resolve="TargetLanguage" />
</node> </node>
<node concept="1TJgyj" id="7kypvuIwECH" role="1TKVEi"> <node concept="1TJgyj" id="7kypvuIwECH" role="1TKVEi">
<property role="IQ2ns" value="8982541288447632591" /> <property role="IQ2ns" value="8982541288447632591" />

44
languages/SemAnno/models/SemAnno.textGen.mps
View File

@@ -41,11 +41,9 @@
<concept id="1068580123136" name="jetbrains.mps.baseLanguage.structure.StatementList" flags="sn" stub="5293379017992965193" index="3clFbS"> <concept id="1068580123136" name="jetbrains.mps.baseLanguage.structure.StatementList" flags="sn" stub="5293379017992965193" index="3clFbS">
<child id="1068581517665" name="statement" index="3cqZAp" /> <child id="1068581517665" name="statement" index="3cqZAp" />
</concept> </concept>
<concept id="1081773326031" name="jetbrains.mps.baseLanguage.structure.BinaryOperation" flags="nn" index="3uHJSO"> <concept id="1081516740877" name="jetbrains.mps.baseLanguage.structure.NotExpression" flags="nn" index="3fqX7Q">
<child id="1081773367579" name="rightExpression" index="3uHU7w" /> <child id="1081516765348" name="expression" index="3fr31v" />
<child id="1081773367580" name="leftExpression" index="3uHU7B" />
</concept> </concept>
<concept id="1073239437375" name="jetbrains.mps.baseLanguage.structure.NotEqualsExpression" flags="nn" index="3y3z36" />
</language> </language>
<language id="b83431fe-5c8f-40bc-8a36-65e25f4dd253" name="jetbrains.mps.lang.textGen"> <language id="b83431fe-5c8f-40bc-8a36-65e25f4dd253" name="jetbrains.mps.lang.textGen">
<concept id="8931911391946696733" name="jetbrains.mps.lang.textGen.structure.ExtensionDeclaration" flags="in" index="9MYSb" /> <concept id="8931911391946696733" name="jetbrains.mps.lang.textGen.structure.ExtensionDeclaration" flags="in" index="9MYSb" />
@@ -68,6 +66,12 @@
<concept id="1233749247888" name="jetbrains.mps.lang.textGen.structure.GenerateTextDeclaration" flags="in" index="11bSqf" /> <concept id="1233749247888" name="jetbrains.mps.lang.textGen.structure.GenerateTextDeclaration" flags="in" index="11bSqf" />
</language> </language>
<language id="7866978e-a0f0-4cc7-81bc-4d213d9375e1" name="jetbrains.mps.lang.smodel"> <language id="7866978e-a0f0-4cc7-81bc-4d213d9375e1" name="jetbrains.mps.lang.smodel">
<concept id="4705942098322609812" name="jetbrains.mps.lang.smodel.structure.EnumMember_IsOperation" flags="ng" index="21noJN">
<child id="4705942098322609813" name="member" index="21noJM" />
</concept>
<concept id="4705942098322467729" name="jetbrains.mps.lang.smodel.structure.EnumMemberReference" flags="ng" index="21nZrQ">
<reference id="4705942098322467736" name="decl" index="21nZrZ" />
</concept>
<concept id="1138056022639" name="jetbrains.mps.lang.smodel.structure.SPropertyAccess" flags="nn" index="3TrcHB"> <concept id="1138056022639" name="jetbrains.mps.lang.smodel.structure.SPropertyAccess" flags="nn" index="3TrcHB">
<reference id="1138056395725" name="property" index="3TsBF5" /> <reference id="1138056395725" name="property" index="3TsBF5" />
</concept> </concept>
@@ -731,15 +735,19 @@
</node> </node>
</node> </node>
</node> </node>
<node concept="3y3z36" id="6LZhwXWa02p" role="3clFbw"> <node concept="3fqX7Q" id="2NUNb3qHSy5" role="3clFbw">
<node concept="2OqwBi" id="IF_PY004" role="3uHU7B"> <node concept="2OqwBi" id="2NUNb3qHSy7" role="3fr31v">
<node concept="117lpO" id="IF_PY005" role="2Oq$k0" /> <node concept="2OqwBi" id="2NUNb3qHSy8" role="2Oq$k0">
<node concept="3TrcHB" id="IF_PY006" role="2OqNvi"> <node concept="117lpO" id="2NUNb3qHSy9" role="2Oq$k0" />
<node concept="3TrcHB" id="2NUNb3qHSya" role="2OqNvi">
<ref role="3TsBF5" to="k8se:7kypvuIwECI" resolve="targetLanguage" /> <ref role="3TsBF5" to="k8se:7kypvuIwECI" resolve="targetLanguage" />
</node> </node>
</node> </node>
<node concept="Xl_RD" id="6LZhwXWavEO" role="3uHU7w"> <node concept="21noJN" id="2NUNb3qHSyb" role="2OqNvi">
<property role="Xl_RC" value="cpp" /> <node concept="21nZrQ" id="2NUNb3qHSyc" role="21noJM">
<ref role="21nZrZ" to="k8se:6LZhwXW98Rc" resolve="cpp" />
</node>
</node>
</node> </node>
</node> </node>
</node> </node>
@@ -970,15 +978,19 @@
</node> </node>
</node> </node>
</node> </node>
<node concept="3y3z36" id="IF_CPP003" role="3clFbw"> <node concept="3fqX7Q" id="2NUNb3qHRZl" role="3clFbw">
<node concept="2OqwBi" id="IF_CPP004" role="3uHU7B"> <node concept="2OqwBi" id="2NUNb3qHRZn" role="3fr31v">
<node concept="117lpO" id="IF_CPP005" role="2Oq$k0" /> <node concept="2OqwBi" id="2NUNb3qHRZo" role="2Oq$k0">
<node concept="3TrcHB" id="IF_CPP006" role="2OqNvi"> <node concept="117lpO" id="2NUNb3qHRZp" role="2Oq$k0" />
<node concept="3TrcHB" id="2NUNb3qHRZq" role="2OqNvi">
<ref role="3TsBF5" to="k8se:7kypvuIwECI" resolve="targetLanguage" /> <ref role="3TsBF5" to="k8se:7kypvuIwECI" resolve="targetLanguage" />
</node> </node>
</node> </node>
<node concept="Xl_RD" id="IF_CPP007" role="3uHU7w"> <node concept="21noJN" id="2NUNb3qHRZr" role="2OqNvi">
<property role="Xl_RC" value="python" /> <node concept="21nZrQ" id="2NUNb3qHRZs" role="21noJM">
<ref role="21nZrZ" to="k8se:6LZhwXW979d" resolve="python" />
</node>
</node>
</node> </node>
</node> </node>
</node> </node>