whetstone_DSL/specialists/ROADMAP.md

Whetstone Specialist Fleet — Roadmap

Read README.md first. This document assumes you understand the core premise.

Guiding principle: "LLM required" is not a permanent category. Every decision is either deterministic, statistical, or not yet decomposed. This roadmap is the plan for collapsing the "not yet decomposed" pile, one specialist at a time.

Architectural correction (important): Whetstone is NOT a C++ code generator. C++, Python, Rust, and all other output languages are projections of an AST-as-source-of-truth. The AST uses language-neutral conceptType names (Function, ClassDeclaration, IfStatement, etc.) that map uniformly across 19+ target languages. Specialists operate at the AST level — one specialist per decision type covers all projection languages. Do not train per-language variants of decisions that live in the AST layer.

---

The AST Is Already Rich

Whetstone's AST is not a minimal syntax tree. It carries annotation metadata that most derived ASTs never have. Key files:

• editor/src/ast/ASTNode.h — base node with conceptType, semanticHash, role-based children
• editor/src/ast/Annotation.h — 60+ annotation types (see Subject breakdown below)
• editor/src/SemanticCoreIR.h — cross-language IR with intentTags
• editor/src/SemanticTags.h — 10-tag vocabulary (@serialize, @crypto, @io, etc.)
• editor/src/ContextAPI.h — CallHierarchyInfo with callerIds/calleeIds
• editor/src/graduation/BlastRadiusAnalyzer.h — impact scoring
• editor/src/DependencyGraph.h — dependency edges and critical path

The metadata that matters for specialists

Downstream impact — already computed, available as input features:

RiskAnnotation.dependentCount    int    how many callers/consumers
CallHierarchyInfo.callerIds      list   which functions call this one
BlastRadiusReport.riskLevel      enum   none/low/medium/high
BlastRadiusEntry.impactScore     int    numerical impact
FailureTriageScore.blastRadius   int    propagation count

Intent and complexity — already stored on nodes:

IntentAnnotation.category        enum   validation/transformation/io/
                                        coordination/computation/initialization
IntentAnnotation.summary         text   1-sentence description
ComplexityAnnotation.timeComplexity     O(1)/O(n)/O(n^2)/...
ComplexityAnnotation.cognitiveComplexity  1-10 scale
SemanticTagAnnotation.tags       list   @serialize/@crypto/@io/@math/etc.

The Tacit Codex annotation — already in the schema:

AutomatabilityAnnotation.strategy  enum   deterministic | template | slm | llm | human
AutomatabilityAnnotation.confidence  float  0.0–1.0

This annotation encodes the entire Tacit Codex hierarchy on individual AST nodes. A specialist that predicts automatability.strategy for a given node is the entropy classifier the Tacit Codex is building toward — it just needs to be trained.

---

Phase 0 — Foundation (COMPLETE)

Done:

• verification_type specialist: 5-way, 83.7% accuracy
  • Input: sprint step description text
  • Output: unit / integration / schema / smoke / docs
  • Checkpoint: /mnt/storage/fabricate_runs/whetstone_verification_type/
• confidence_tier specialist: 3-way, 100% accuracy
  • Input: "conflicts=N ambiguity=M deps=D prereqs=P queueready=yes/no"
  • Output: high / medium / low
  • Checkpoint: /mnt/storage/fabricate_runs/whetstone_confidence_tier/
• Inference wrapper: scripts/infer.py

Key finding from Phase 0: Sprint plan step descriptions are the right training data source for text-based specialists. The 18,519 JSONL pipeline run records are too homogeneous (2 task patterns repeated). Do not use JSONL as the text source.

---

Phase 1 — Pipeline Specialists (NEXT)

Goal: Cover the remaining bounded decisions in generate_taskitems and validate_taskitem. These are at the taskitem planning layer, above the AST.

1.1 — prereq_op_selector (4-way)

Input: task step description text Output:

0: standard       → [validate-intake, resolve-dependencies]
1: needs_review   → [validate-intake, architect-review]
2: needs_approval → [validate-intake, manual-approval]
3: full_gates     → [validate-intake, architect-review, manual-approval]

Training data: synthetic — generate task descriptions that signal each gate type. Script to write: scripts/gen_prereq_op_data.py

1.2 — worker_type (4-way)

Input: task step description text Output: implementer / reviewer / architect / qa Training data: sprint plan files (same heuristic approach as verification_type). Script to write: scripts/gen_worker_type_data.py

1.3 — gap_class (5-way)

Input: taskitem issue description text Output: under_constrained / environment_gap / planning_gap / cross_project_gap / generator_capability_gap Training data: synthetic — generate issue descriptions encoding each gap type. Script to write: scripts/gen_gap_class_data.py

1.4 — escalation_predictor (binary)

Input: "confidence=N ambiguity=M" structural encoding Output: escalate / proceed Training data: synthetic from formula (escalate = confidence < 60 OR ambiguity > 0). Expected accuracy: 100% (deterministic formula, same as confidence_tier). Script to write: scripts/gen_escalation_data.py

---

Phase 2 — AST Annotation Specialists

Goal: Train specialists that predict annotation fields on AST nodes. These operate one level deeper than the taskitem pipeline — they answer questions about individual AST nodes, not about sprint planning.

The annotation system already defines the output schema. Specialists populate annotation fields that are currently filled by the LLM or left empty.

2.1 — automatability_strategy specialist (THE KEY ONE) — (6-way)

This is the Tacit Codex entropy classifier applied to AST nodes.

IMPORTANT — update editor/src/ast/Annotation.h first. The current enum (deterministic|template|slm|llm|human) is missing the specialist tier. The correct 6-tier hierarchy is:

deterministic   same input → same output always. Formula. Zero parameters.
template        deterministic composition of known structural patterns. No ambiguity.
specialist      bounded output vocabulary (fixed N classes). Fuzzy/probabilistic.
                ~800KB model, microseconds — THIS PROJECT's outputs.
slm             open vocabulary, arbitrary token generation. ~1-7GB.
llm             full generative, cross-context, novel reasoning. ~14B+.
human           not yet formalizable.

Boundary between specialist and slm: can the output be a fixed enum? Yes → specialist. Requires generating novel tokens → slm or llm. Boundary between template and specialist: deterministic? Yes → template. Probabilistic with a confidence score → specialist.

Add "specialist" to the strategy field in Annotation.h before training.

Input features (all from existing AST/annotation metadata):

conceptType          (Function/Class/Variable/EnumDeclaration/...)
intentCategory       (validation/transformation/io/coordination/computation/init)
dependentCount       (0-100+ from RiskAnnotation)
semanticTags         (space-separated: @io @serialize @math...)
cognitiveComplexity  (1-10 from ComplexityAnnotation)
hasBody              (yes/no — skeleton vs implemented)

Encoded as: "type=Function intent=validation deps=12 tags=@io @serialize complexity=4 body=yes"

Output: deterministic / template / specialist / slm / llm / human

Why this is the most important specialist: Once trained, every node in the AST can be tagged with its automatability tier. The pipeline routes each node's decisions to the appropriate handler:

• deterministic → formula/rule, no inference at all
• template → template instantiation, near-zero cost
• specialist → specialist fleet dispatch (~0.5ms)
• slm → small model call (~500ms)
• llm → full LLM call (~2-5s)
• human → escalate via ReviewAnnotation

This replaces the current situation where every decision defaults to "llm" regardless.

Training data: The 93 existing editor/src/*.h headers are labeled examples. Each header class can be manually categorized. Starting corpus: ~93 examples (thin but the patterns are strong — expect grokking with heavy oversampling). Future: every new header added to the codebase extends the training set.

Script to write: scripts/gen_automatability_data.py

2.2 — intent_category specialist (6-way)

Input: node name + semantic tags + parent conceptType (all language-neutral) Output: validation / transformation / io / coordination / computation / initialization

Training data: mine existing headers. The class name + role suffix strongly predicts the category:

• *Validator, *Scorer, *Auditor → validation
• *Generator, *Builder, *Composer → transformation
• *Loader, *Writer, *Exporter → io
• *Dispatcher, *Router, *Registry → coordination
• *Calculator, *Analyzer, *Solver → computation
• *Initializer, *Factory, *Bootstrap → initialization

This is trainable from class name patterns alone. 93 headers = 93 examples to start. Script to write: scripts/gen_intent_category_data.py

2.3 — change_risk_tier specialist (4-way)

Input: "deps=N callers=M impactScore=K blastRadius=R" (all from existing metadata) Output: low / medium / high / critical

This specialist uses the blast radius and downstream dependency counts as its primary input signal. A function with deps=47 and blastRadius=12 is high risk. A function with deps=0 and callers=1 is low risk.

Training data: synthetic from rules:

• critical: deps > 20 OR callerIds.count > 15
• high: deps 10-20 OR callers 8-15
• medium: deps 3-10 OR callers 2-8
• low: deps 0-3, callers 0-2

Script to write: scripts/gen_change_risk_data.py Expected accuracy: ~100% (formula-based, same as confidence_tier).

2.4 — semantic_tag specialist (multi-label → primary tag, 10-way)

Input: node name text (language-neutral conceptual name) Output: primary semantic tag from vocabulary: @serialize / @crypto / @io / @network / @math / @collection / @concurrency / @test / @ui / @parse

The SemanticTags.h file already has auto-tagging rules based on library names and identifier keywords. This specialist learns the same rules from examples.

Training data: mine existing headers using the auto-tagger rules as labels. Script to write: scripts/gen_semantic_tag_data.py

2.5 — component_suffix specialist (12-way)

Input: 1-sentence description of what a component does Output: naming suffix that fits the role: Validator / Scorer / Auditor / Generator / Processor / Selector / Advisor / Binder / Extractor / Composer / Builder / Registry

Training data: 93 existing editor/src/*.h headers as labeled examples. Each filename encodes the ground-truth suffix. Thin data but strong patterns. Script to write: scripts/gen_component_suffix_data.py

---

Phase 3 — Data Collection (Running in Background)

Goal: Make every LLM tool call a training data generator.

3.1 — Instrument bounded sub-decisions

Add post-processing to tools/mcp/export_taskitem_run_for_lora.sh to extract per-decision labels from each pipeline run. When generate_taskitems emits a task with verificationType: "integration", append (step_description, "integration") to training_data/specialists/verification_type_examples.jsonl.

3.2 — Annotation capture hook

When the LLM fills an AutomatabilityAnnotation or IntentAnnotation on any AST node, capture (input_features, annotation_value) as a training example. These accumulate automatically with normal usage.

3.3 — Blast radius instrumentation

When BlastRadiusAnalyzer runs, emit (dependentCount, blastRadius, callerCount) → riskLevel as a labeled example. This grows the change_risk_tier training set with every real analysis run.

---

Phase 4 — Integration

Goal: Wire specialists into the MCP tool layer.

4.1 — Dispatch server

scripts/dispatch.py — loads all trained specialists at startup, accepts JSON, returns predictions. All specialists run in a single batched forward pass.

4.2 — Pre-annotation pass

New MCP tool: whetstone_annotate_nodes — given a list of AST node descriptors, runs the specialist fleet and returns predicted annotation values before the LLM is called. The LLM then only handles open-text fields and human-tier decisions.

4.3 — Automatability routing

Once automatability_strategy specialist is trained and validated:

• deterministic nodes: handled by formula/rule engine, no inference
• template nodes: handled by template instantiation
• slm nodes: handled by specialist dispatch
• llm nodes: LLM call with pre-filled context from specialists
• human nodes: escalated via ReviewAnnotation.required = true

---

Phase 5 — Code Generation Specialists

Goal: Decompose "generative" decisions into bounded sub-decisions.

Once Phase 2 establishes the full annotation vocabulary, code generation becomes:

Node with annotations
    → automatability_strategy = "template"         ← specialist decides
    → intent_category = "transformation"           ← specialist decides
    → structural_pattern = "pipeline"              ← specialist decides (new)
    → control_flow_pattern = "early-return"        ← specialist decides (new)
    → error_handling_style = "Result<T,E>"         ← specialist decides (new)
    → projection language = "cpp"                  ← human or router decides
    ↓
Deterministic template instantiation
    ↓
Code skeleton
    ↓
LLM fills only: identifier names, literal values, comments  ← narrow residual

Specialists for this phase:

• structural_pattern (factory/visitor/strategy/builder/observer/pipeline/scorer/registry...)
• control_flow_pattern (linear/if-else-chain/early-return/loop/recursive/dispatch-table...)
• error_handling_style (exception/Result/optional/expected/assert/error-code...)
• data_structure_choice (vector/map/deque/set/flat-array/ring-buffer...)

Each of these is a bounded classification over the AST node's role and annotation context. Each can be trained from the existing editor/src/ headers as examples.

---

Tracking Progress

Specialist	Phase	Status	Accuracy	Checkpoint
verification_type	0	✅ DONE	83.7%	/mnt/storage/fabricate_runs/whetstone_verification_type/
confidence_tier	0	✅ DONE	100%	/mnt/storage/fabricate_runs/whetstone_confidence_tier/
prereq_op_selector	1.1	⬜ planned	—	—
worker_type	1.2	⬜ planned	—	—
gap_class	1.3	⬜ planned	—	—
escalation_predictor	1.4	⬜ planned	—	—
automatability_strategy	2.1	⬜ planned (6-way, needs Annotation.h update first)	—	—
intent_category	2.2	⬜ planned	—	—
change_risk_tier	2.3	⬜ planned	—	—
semantic_tag	2.4	⬜ planned	—	—
component_suffix	2.5	⬜ planned	—	—
structural_pattern	5	⬜ planned	—	—
control_flow_pattern	5	⬜ planned	—	—
error_handling_style	5	⬜ planned	—	—

---

What the Next Agent Should Do First

1. Read README.md and this file completely
2. Verify Phase 0 specialists by running infer.py --eval-tsv on both held-out sets
3. Start Phase 2.1 (automatability_strategy) — it is the most architecturally significant specialist and training data already exists in editor/src/*.h
4. Then Phase 1.1 (prereq_op_selector) to complete the pipeline layer
5. Do not train per-language specialist variants — the AST is language-neutral; projection is handled downstream by deterministic grammar rules

---

Connection to Other Projects

WhetstoneAI_Fabricate (/home/bill/Documents/WhetstoneAI_Fabricate/): Training infrastructure. Binary, harness, eval tools. The RSA experiments (see HANDOFF-2026-03-28-RSA.md) validated the specialist architecture at scale. The label convention fix (swap_label_rows.py) is documented there — read it before resuming any checkpoint with a changed --labels order.

Tacit Codex (tacit_codex/): AutomatabilityAnnotation.strategy in editor/src/ast/Annotation.h is the Tacit Codex entropy classification built into the AST schema. Training the automatability_strategy specialist is the concrete implementation of the Tacit Codex collapse-to-formal-structure program applied to individual AST nodes. The entropy ledger in HiveMind tracks the same quantity at the job-scheduling layer.

HiveMind (/home/bill/Documents/hivemind/): Future state: a generate_taskitems job becomes a fan-out of N specialist jobs

• 1 LLM job (only for llm-tier nodes), merged by coordinator. The automatability tier determines which job type each node gets dispatched to.