114 lines
4.8 KiB
Markdown
114 lines
4.8 KiB
Markdown
|
# Whetstone DSL — Session Handoff
|
|||
|
|
**Date:** 2026-03-02
|
|||
|
|
**Session:** Phase 6 Planning
|
|||
|
|
**Last step completed:** 1957 (Sprint 285 — poly-everything)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## State on Entry
|
|||
|
|
|
|||
|
|
Sprints 284 and 285 were completed at the start of this session:
|
|||
|
|
- Sprint 284 (1948–1952): poly-compiler — CompilerPhaseSpec, CompilerIRNode, CompilerPhaseRouter, CompilerPipelineRunner
|
|||
|
|
- Sprint 285 (1953–1957): poly-everything — PolyglotFunctionMap, PolyglotExecutionContext, PolyglotResultAggregator, PolyglotOrchestrator
|
|||
|
|
- Phase 5 (Sprints 282–285) fully complete
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Key Decisions Made This Session
|
|||
|
|
|
|||
|
|
### 1. Gap Registry: Clean
|
|||
|
|
All 26 GR-series gaps closed as of 2026-02-28. Compatibility ledger has one resolved entry. Nothing pending.
|
|||
|
|
|
|||
|
|
### 2. Phase 6: Library Dispatch
|
|||
|
|
The system currently chooses **languages** but not **libraries**. This is the central gap: "write a matrix multiply in C++" and "use cublasSgemm" are not the same instruction.
|
|||
|
|
|
|||
|
|
Phase 6 closes this by making library + API selection **fully deterministic**:
|
|||
|
|
- `OperationTaxonomy` — fixed registry of operation domains (serialization.json, compute.matrix, etc.)
|
|||
|
|
- `LibraryCapabilityLedger` — per (library, operationDomain): score + known weaknesses + preferred APIs
|
|||
|
|
- `PerTaskLibrarySelector` — deterministic max-score dispatch per task
|
|||
|
|
- `LibrarySymbolAdvisor` — specific API function per (library, operationDomain)
|
|||
|
|
- Integration into `generate_taskitems` — execution contracts gain selectedLibrary, operationDomain, preferredAPIs
|
|||
|
|
|
|||
|
|
No LLM or human in the selection loop at runtime. Annotation is one-time authoring.
|
|||
|
|
|
|||
|
|
**Key insight:** At deterministic compute speeds, "combinatorial explosion" across libraries × operations is irrelevant — the entire ledger evaluates in microseconds. We have essentially unlimited budget relative to what an LLM inference costs.
|
|||
|
|
|
|||
|
|
### 3. Per-Task Granularity (Not Per-Project)
|
|||
|
|
Library selection is per-operation-domain, per-task — not per-project stack. A project using protobuf for IPC still selects nlohmann_json for a JSON serialization task (0.92 vs 0.60). The library that handles an operation best wins that task regardless of project stack.
|
|||
|
|
|
|||
|
|
### 4. Training Data Assessment
|
|||
|
|
- **Total:** ~517K entries, 4.2 GB
|
|||
|
|
- **Core signal:** mcp_calls_bad.jsonl (16K, failure modes), taskitem_pipeline_runs.jsonl (18.5K, pipeline structure), datasets/ benchmarks
|
|||
|
|
- **Will become wrong after Phase 6:** The bulk of mcp_calls_good.jsonl and mcp_call_quality.jsonl — these train the SLM to make decisions that Phase 6 moves to deterministic compute. Training on them post-Phase-6 teaches wrong behavior.
|
|||
|
|
- **Action:** Archive bad calls + pipeline structure + benchmark specs (~250 MB). Retire good call quality data (~350 MB) after Phase 6 is live. Discard logs/ execution traces (1.5 GB noise).
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## What to Do Next
|
|||
|
|
|
|||
|
|
### Immediate: Start Sprint 286
|
|||
|
|
|
|||
|
|
Plan doc: `docs/phase6_library_dispatch_sprint_plan.md`
|
|||
|
|
|
|||
|
|
Sprint 286 (steps 1958–1962): OperationTaxonomy + LibraryCapabilityLedger
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
start_recording (session_id: sprint286-2026-03-02)
|
|||
|
|
→ architect_intake (use phase6 plan Sprint 286 section as spec)
|
|||
|
|
→ generate_taskitems
|
|||
|
|
→ queue_ready
|
|||
|
|
→ run_pipeline ×4
|
|||
|
|
→ implement headers:
|
|||
|
|
OperationDomain.h
|
|||
|
|
LibraryCapabilityRecord.h
|
|||
|
|
LibraryCapabilityLedger.h
|
|||
|
|
OperationTaxonomy.h
|
|||
|
|
Sprint286IntegrationSummary.h
|
|||
|
|
→ write tests: step1958_test.cpp – step1962_test.cpp
|
|||
|
|
→ update CMakeLists (entries for step1958–step1962)
|
|||
|
|
→ build + run tests
|
|||
|
|
→ get_metrics → commit → update CLAUDE.md + MEMORY.md → write handoff
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
Initial library seeds for step1962 integration test:
|
|||
|
|
- nlohmann_json: serialization.json=0.92
|
|||
|
|
- protobuf: serialization.binary=0.95, serialization.json=0.60
|
|||
|
|
- serde_json: serialization.json=0.98
|
|||
|
|
- cublas: compute.matrix=0.99
|
|||
|
|
- thrust: compute.parallel=0.97
|
|||
|
|
- numpy: compute.matrix=0.95, compute.statistics=0.93
|
|||
|
|
- tokio: network.async=0.97
|
|||
|
|
- boost.asio: network.async=0.91
|
|||
|
|
|
|||
|
|
### Sprint Sequence
|
|||
|
|
| Sprint | Steps | Theme |
|
|||
|
|
|--------|-------|-------|
|
|||
|
|
| 286 | 1958–1962 | OperationTaxonomy + LibraryCapabilityLedger |
|
|||
|
|
| 287 | 1963–1967 | PerTaskLibrarySelector |
|
|||
|
|
| 288 | 1968–1972 | LibrarySymbolAdvisor |
|
|||
|
|
| 289 | 1973–1977 | Integration into generate_taskitems |
|
|||
|
|
| 290 | 1978–1982 | CUDA end-to-end proof |
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Files Written This Session
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
docs/phase6_library_dispatch_sprint_plan.md — full Phase 6 plan (Sprints 286–290)
|
|||
|
|
HANDOFF-2026-03-02-phase6-plan.md — this file
|
|||
|
|
CLAUDE.md — updated to Phase 6
|
|||
|
|
MEMORY.md — updated to Sprint 285 complete, Phase 6 next
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
No code written this session — planning only.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
## Build Status
|
|||
|
|
|
|||
|
|
All tests passing through step1957. Branch: `001-core-ast-structure`.
|
|||
|
|
```bash
|
|||
|
|
cmake --build editor/build-native --parallel # builds all targets
|
|||
|
|
./editor/build-native/step1957_test # 5/5 pass
|
|||
|
|
```
|