143e9014b3
All 5 sprints (286-290, steps 1958-1982) complete. 50/50 tests passing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
176 lines
7.0 KiB
Markdown
176 lines
7.0 KiB
Markdown
# Whetstone DSL — Claude Code Session Guide
|
||
|
||
> Read this before starting any work on the Whetstone editor.
|
||
> This project is independent of HiveMind. Do not conflate them.
|
||
|
||
---
|
||
|
||
## What Whetstone Is
|
||
|
||
Whetstone is a **general-purpose constructive editor and MCP server** for
|
||
cross-language code generation. It is NOT a HiveMind-specific tool — HiveMind
|
||
is one of many projects that consumes it. Whetstone's core capabilities:
|
||
|
||
- Semantic annotation DSL (SemAnno): annotate code with memory/ownership intent
|
||
- 19+ language parsers and generators (Python, C++, Rust, Go, JS/TS, Java, and more)
|
||
- Cross-language transpilation (AST-level, annotation-guided)
|
||
- 93+ MCP tools callable by any MCP client over stdio
|
||
- Headless `whetstone_mcp` binary + full ImGui GUI editor (`whetstone_editor`)
|
||
|
||
**Stack:** C++20, Dear ImGui + SDL2 + OpenGL3, nlohmann-json, tree-sitter, vcpkg, CMake
|
||
|
||
---
|
||
|
||
## Current State
|
||
|
||
| Item | Value |
|
||
|------|-------|
|
||
| Last step | **Step 1982** |
|
||
| Last sprint | **Sprint 290 — COMPLETE** |
|
||
| MCP tool count | **93 tools** |
|
||
| Architecture gate | All headers ≤ 600 lines (pre-existing BufferOps.h violation) |
|
||
| `whetstone_mcp` binary | `editor/build-native/whetstone_mcp` |
|
||
| Active track | **Phase 6 COMPLETE** — Next: Phase 7 (TBD) |
|
||
|
||
Sprint history: see `progress.md` (15k+ lines) — single authoritative file,
|
||
sprint summary table at top, step-by-step detail below.
|
||
|
||
Phase 6 plan (`docs/phase6_library_dispatch_sprint_plan.md`) **FULLY COMPLETE**:
|
||
- Sprint 286 (1958–1962): OperationTaxonomy + LibraryCapabilityLedger — COMPLETE
|
||
- Sprint 287 (1963–1967): PerTaskLibrarySelector — COMPLETE
|
||
- Sprint 288 (1968–1972): LibrarySymbolAdvisor — COMPLETE
|
||
- Sprint 289 (1973–1977): Integration into generate_taskitems — COMPLETE
|
||
- Sprint 290 (1978–1982): CUDA End-to-End Proof — COMPLETE
|
||
|
||
Phase 5 plan (`docs/polyglot_orchestrator_sprint_plan.md`) fully executed:
|
||
- Sprint 282 (1938–1942): Polyglot Test Harness — COMPLETE
|
||
- Sprint 283 (1943–1947): poly-pipeline — COMPLETE
|
||
- Sprint 284 (1948–1952): poly-compiler — COMPLETE
|
||
- Sprint 285 (1953–1957): poly-everything — COMPLETE
|
||
|
||
---
|
||
|
||
## Sprint Workflow (TDD Pattern)
|
||
|
||
Every sprint follows this exact pattern:
|
||
1. `start_recording` → `architect_intake` → `generate_taskitems` → `queue_ready` → `run_pipeline ×4`
|
||
2. Implement headers in `editor/src/` (single-class, ≤600 lines)
|
||
3. Write tests in `editor/tests/stepNNN_test.cpp`
|
||
4. Add CMakeLists entries, build, run all 5 tests
|
||
5. `get_metrics` → commit → update CLAUDE.md + MEMORY.md → write handoff
|
||
|
||
**Phase 6 COMPLETE.** All 5 sprints (286–290, steps 1958–1982) done. 50/50 tests passing.
|
||
|
||
Phase 6 delivered deterministic per-task library + API dispatch:
|
||
- Sprint 286 (1958–1962): OperationTaxonomy + LibraryCapabilityLedger — **COMPLETE**
|
||
- Sprint 287 (1963–1967): PerTaskLibrarySelector — **COMPLETE**
|
||
- Sprint 288 (1968–1972): LibrarySymbolAdvisor — **COMPLETE**
|
||
- Sprint 289 (1973–1977): Integration into generate_taskitems — **COMPLETE**
|
||
- Sprint 290 (1978–1982): CUDA end-to-end proof — **COMPLETE**
|
||
|
||
**Next: Phase 7 (TBD).** Start a new session to plan the next phase.
|
||
|
||
---
|
||
|
||
## Build
|
||
|
||
```bash
|
||
# Native build (this machine: Linux x86_64)
|
||
cd /home/bill/Documents/CLionProjects/whetstone_DSL
|
||
cmake --build editor/build-native --target whetstone_mcp --parallel
|
||
|
||
# Build the GUI editor too
|
||
cmake --build editor/build-native --target whetstone_editor --parallel
|
||
|
||
# Run a specific step test
|
||
./editor/build-native/step1887_test
|
||
|
||
# Run the architecture gate
|
||
./editor/build-native/file_limits_test
|
||
|
||
# Full rebuild from scratch
|
||
cd editor
|
||
cmake -S . -B build-native -DCMAKE_BUILD_TYPE=Release \
|
||
-DCMAKE_TOOLCHAIN_FILE=/home/bill/vcpkg/scripts/buildsystems/vcpkg.cmake
|
||
cmake --build build-native --parallel
|
||
```
|
||
|
||
**vcpkg packages** (already installed in `editor/build-native/vcpkg_installed/`):
|
||
nlohmann-json, sdl2, imgui, glad, tree-sitter, and language grammars via FetchContent.
|
||
|
||
---
|
||
|
||
## Key Directories
|
||
|
||
```
|
||
editor/
|
||
src/ C++ source — all components as single-class headers
|
||
src/mcp/ MCP tool registration headers (Register*.h)
|
||
src/ast/ AST nodes, parsers, generators
|
||
src/state/ EditorState sub-states
|
||
src/panels/ ImGui panel components
|
||
tests/ One test file per step (stepNNN_test.cpp)
|
||
build-native/ Build output (do not commit binaries)
|
||
tools/
|
||
claude/
|
||
tools.json MCP tool catalog (update when adding new tools)
|
||
```
|
||
|
||
---
|
||
|
||
## Adding a New MCP Tool (Pattern)
|
||
|
||
1. Implement the C++ class in `editor/src/MyFeature.h` (≤ 600 lines)
|
||
2. Create `editor/src/mcp/RegisterMyFeatureTools.h`:
|
||
- Define tool name, input schema, handler lambda
|
||
- Handler calls `MyFeature::run(args)` and returns JSON result
|
||
3. Add `#include "mcp/RegisterMyFeatureTools.h"` to `RegisterOnboardingAndAllTools.h`
|
||
4. Add entry to `tools/claude/tools.json`
|
||
5. Rebuild `whetstone_mcp` and verify tool appears via MCP initialize response
|
||
|
||
Reference implementations: `RegisterCodegenTools.h`, `RegisterContextTools.h`,
|
||
`RegisterValidationTools.h`, `RegisterMetricsTools.h`
|
||
|
||
---
|
||
|
||
## MCP Tools by Sprint (Key Tools)
|
||
|
||
| Tool | Sprint | What It Does |
|
||
|------|--------|-------------|
|
||
| `whetstone_architect_intake` | 36 | Markdown spec → normalized requirements + conflict report |
|
||
| `whetstone_generate_taskitems` | 36 | Requirements → AnnotatedTaskitems with confidence/escalate |
|
||
| `whetstone_queue_ready` | 36 | Validate taskitem set → confirmed queue |
|
||
| `whetstone_schema_to_cpp` | 41 | JSON Schema → typed C++ header + CMake snippet |
|
||
| `whetstone_generate_dispatch_table` | 41 | Job entries → C++ dispatch table header |
|
||
| `whetstone_assemble_context` | 43 | Workspace symbol lookup → token-budget-trimmed context slice |
|
||
| `whetstone_validate_taskitem` | 44 | Score taskitem self-containment (0–100), batch audit report |
|
||
| `whetstone_start_recording` | 45 | Start per-session tool-call recording |
|
||
| `whetstone_get_metrics` | 45 | Get session metrics + A/B comparison vs baseline |
|
||
| `whetstone_score_language_fitness` | 271 | Score AST features against language profiles → ranked list |
|
||
|
||
Full tool list: `tools/claude/tools.json` (91 entries).
|
||
|
||
---
|
||
|
||
## Architecture Constraints
|
||
|
||
- **No header may exceed 600 lines** — enforced by `file_limits_test`
|
||
- Single-class-per-header pattern throughout
|
||
- MCP tools are thin wiring only — business logic lives in the feature class
|
||
- `tools.json` must be updated every time a tool is added/removed
|
||
- Never touch `whetstone_mcp` for GUI-only features; keep headless binary lean
|
||
|
||
---
|
||
|
||
## Relationship to HiveMind
|
||
|
||
Whetstone generates code for HiveMind (schema→C++ structs, dispatch tables).
|
||
HiveMind is a **consumer** of the MCP tools. The two projects are independent:
|
||
|
||
- **Whetstone source:** `CLionProjects/whetstone_DSL/` — this directory
|
||
- **HiveMind source:** `Documents/hivemind/` — separate project
|
||
- Do NOT modify Whetstone when working on HiveMind, and vice versa
|
||
|
||
Whetstone MCP is registered at `Documents/.mcp.json` so it's available
|
||
as a tool server in Claude Code sessions opened from `Documents/`.
|