# Whetstone Specialist Fleet — Roadmap

**Read README.md first.** This document assumes you understand the core premise.

**Guiding principle:** "LLM required" is not a permanent category. Every decision is
either deterministic, statistical, or not yet decomposed. This roadmap is the plan
for collapsing the "not yet decomposed" pile, one specialist at a time.

**Architectural correction (important):** Whetstone is NOT a C++ code generator.
C++, Python, Rust, and all other output languages are *projections* of an
AST-as-source-of-truth. The AST uses language-neutral `conceptType` names
(`Function`, `ClassDeclaration`, `IfStatement`, etc.) that map uniformly across
19+ target languages. Specialists operate at the AST level — one specialist per
decision type covers all projection languages. Do not train per-language variants
of decisions that live in the AST layer.

---

## The AST Is Already Rich

Whetstone's AST is not a minimal syntax tree. It carries annotation metadata that
most derived ASTs never have. Key files:

- `editor/src/ast/ASTNode.h` — base node with `conceptType`, `semanticHash`, role-based children
- `editor/src/ast/Annotation.h` — 60+ annotation types (see Subject breakdown below)
- `editor/src/SemanticCoreIR.h` — cross-language IR with `intentTags`
- `editor/src/SemanticTags.h` — 10-tag vocabulary (`@serialize`, `@crypto`, `@io`, etc.)
- `editor/src/ContextAPI.h` — `CallHierarchyInfo` with `callerIds`/`calleeIds`
- `editor/src/graduation/BlastRadiusAnalyzer.h` — impact scoring
- `editor/src/DependencyGraph.h` — dependency edges and critical path

### The metadata that matters for specialists

**Downstream impact** — already computed, available as input features:
```
RiskAnnotation.dependentCount    int    how many callers/consumers
CallHierarchyInfo.callerIds      list   which functions call this one
BlastRadiusReport.riskLevel      enum   none/low/medium/high
BlastRadiusEntry.impactScore     int    numerical impact
FailureTriageScore.blastRadius   int    propagation count
```

**Intent and complexity** — already stored on nodes:
```
IntentAnnotation.category        enum   validation/transformation/io/
                                        coordination/computation/initialization
IntentAnnotation.summary         text   1-sentence description
ComplexityAnnotation.timeComplexity     O(1)/O(n)/O(n^2)/...
ComplexityAnnotation.cognitiveComplexity  1-10 scale
SemanticTagAnnotation.tags       list   @serialize/@crypto/@io/@math/etc.
```

**The Tacit Codex annotation — already in the schema:**
```
AutomatabilityAnnotation.strategy  enum   deterministic | template | slm | llm | human
AutomatabilityAnnotation.confidence  float  0.0–1.0
```

This annotation encodes the entire Tacit Codex hierarchy on individual AST nodes.
A specialist that predicts `automatability.strategy` for a given node *is* the
entropy classifier the Tacit Codex is building toward — it just needs to be trained.

---

## Phase 0 — Foundation (COMPLETE)

**Done:**
- `verification_type` specialist: 5-way, 83.7% accuracy
  - Input: sprint step description text
  - Output: unit / integration / schema / smoke / docs
  - Checkpoint: `/mnt/storage/fabricate_runs/whetstone_verification_type/`
- `confidence_tier` specialist: 3-way, 100% accuracy
  - Input: `"conflicts=N ambiguity=M deps=D prereqs=P queueready=yes/no"`
  - Output: high / medium / low
  - Checkpoint: `/mnt/storage/fabricate_runs/whetstone_confidence_tier/`
- Inference wrapper: `scripts/infer.py`

**Key finding from Phase 0:** Sprint plan step descriptions are the right training
data source for text-based specialists. The 18,519 JSONL pipeline run records are
too homogeneous (2 task patterns repeated). Do not use JSONL as the text source.

---

## Phase 1 — Pipeline Specialists (NEXT)

**Goal:** Cover the remaining bounded decisions in `generate_taskitems`
and `validate_taskitem`. These are at the taskitem planning layer, above the AST.

### 1.1 — `prereq_op_selector` (4-way)

Input: task step description text
Output:
```
0: standard       → [validate-intake, resolve-dependencies]
1: needs_review   → [validate-intake, architect-review]
2: needs_approval → [validate-intake, manual-approval]
3: full_gates     → [validate-intake, architect-review, manual-approval]
```
Training data: synthetic — generate task descriptions that signal each gate type.
Script to write: `scripts/gen_prereq_op_data.py`

### 1.2 — `worker_type` (4-way)

Input: task step description text
Output: implementer / reviewer / architect / qa
Training data: sprint plan files (same heuristic approach as `verification_type`).
Script to write: `scripts/gen_worker_type_data.py`

### 1.3 — `gap_class` (5-way)

Input: taskitem issue description text
Output: under_constrained / environment_gap / planning_gap / cross_project_gap / generator_capability_gap
Training data: synthetic — generate issue descriptions encoding each gap type.
Script to write: `scripts/gen_gap_class_data.py`

### 1.4 — `escalation_predictor` (binary)

Input: `"confidence=N ambiguity=M"` structural encoding
Output: escalate / proceed
Training data: synthetic from formula (`escalate = confidence < 60 OR ambiguity > 0`).
Expected accuracy: 100% (deterministic formula, same as `confidence_tier`).
Script to write: `scripts/gen_escalation_data.py`

---

## Phase 2 — AST Annotation Specialists

**Goal:** Train specialists that predict annotation fields on AST nodes.
These operate one level deeper than the taskitem pipeline — they answer questions
about individual AST nodes, not about sprint planning.

The annotation system already defines the output schema. Specialists populate
annotation fields that are currently filled by the LLM or left empty.

### 2.1 — `automatability_strategy` specialist (THE KEY ONE) — (6-way)

**This is the Tacit Codex entropy classifier applied to AST nodes.**

**IMPORTANT — update `editor/src/ast/Annotation.h` first.** The current enum
(`deterministic|template|slm|llm|human`) is missing the `specialist` tier.
The correct 6-tier hierarchy is:

```
deterministic   same input → same output always. Formula. Zero parameters.
template        deterministic composition of known structural patterns. No ambiguity.
specialist      bounded output vocabulary (fixed N classes). Fuzzy/probabilistic.
                ~800KB model, microseconds — THIS PROJECT's outputs.
slm             open vocabulary, arbitrary token generation. ~1-7GB.
llm             full generative, cross-context, novel reasoning. ~14B+.
human           not yet formalizable.
```

Boundary between `specialist` and `slm`: can the output be a fixed enum?
Yes → specialist. Requires generating novel tokens → slm or llm.
Boundary between `template` and `specialist`: deterministic? Yes → template.
Probabilistic with a confidence score → specialist.

Add `"specialist"` to the strategy field in `Annotation.h` before training.

Input features (all from existing AST/annotation metadata):
```
conceptType          (Function/Class/Variable/EnumDeclaration/...)
intentCategory       (validation/transformation/io/coordination/computation/init)
dependentCount       (0-100+ from RiskAnnotation)
semanticTags         (space-separated: @io @serialize @math...)
cognitiveComplexity  (1-10 from ComplexityAnnotation)
hasBody              (yes/no — skeleton vs implemented)
```
Encoded as: `"type=Function intent=validation deps=12 tags=@io @serialize complexity=4 body=yes"`

Output: deterministic / template / **specialist** / slm / llm / human

**Why this is the most important specialist:**
Once trained, every node in the AST can be tagged with its automatability tier.
The pipeline routes each node's decisions to the appropriate handler:
- `deterministic` → formula/rule, no inference at all
- `template` → template instantiation, near-zero cost
- `specialist` → specialist fleet dispatch (~0.5ms)
- `slm` → small model call (~500ms)
- `llm` → full LLM call (~2-5s)
- `human` → escalate via ReviewAnnotation

This replaces the current situation where every decision defaults to "llm" regardless.

Training data: The 93 existing `editor/src/*.h` headers are labeled examples.
Each header class can be manually categorized. Starting corpus: ~93 examples
(thin but the patterns are strong — expect grokking with heavy oversampling).
Future: every new header added to the codebase extends the training set.

Script to write: `scripts/gen_automatability_data.py`

### 2.2 — `intent_category` specialist (6-way)

Input: node name + semantic tags + parent conceptType (all language-neutral)
Output: validation / transformation / io / coordination / computation / initialization

Training data: mine existing headers. The class name + role suffix strongly predicts
the category:
- `*Validator`, `*Scorer`, `*Auditor` → validation
- `*Generator`, `*Builder`, `*Composer` → transformation
- `*Loader`, `*Writer`, `*Exporter` → io
- `*Dispatcher`, `*Router`, `*Registry` → coordination
- `*Calculator`, `*Analyzer`, `*Solver` → computation
- `*Initializer`, `*Factory`, `*Bootstrap` → initialization

This is trainable from class name patterns alone. 93 headers = 93 examples to start.
Script to write: `scripts/gen_intent_category_data.py`

### 2.3 — `change_risk_tier` specialist (4-way)

Input: `"deps=N callers=M impactScore=K blastRadius=R"` (all from existing metadata)
Output: low / medium / high / critical

This specialist uses the blast radius and downstream dependency counts as its
primary input signal. A function with `deps=47` and `blastRadius=12` is high risk.
A function with `deps=0` and `callers=1` is low risk.

Training data: synthetic from rules:
- critical: deps > 20 OR callerIds.count > 15
- high: deps 10-20 OR callers 8-15
- medium: deps 3-10 OR callers 2-8
- low: deps 0-3, callers 0-2

Script to write: `scripts/gen_change_risk_data.py`
Expected accuracy: ~100% (formula-based, same as `confidence_tier`).

### 2.4 — `semantic_tag` specialist (multi-label → primary tag, 10-way)

Input: node name text (language-neutral conceptual name)
Output: primary semantic tag from vocabulary:
  @serialize / @crypto / @io / @network / @math / @collection /
  @concurrency / @test / @ui / @parse

The `SemanticTags.h` file already has auto-tagging rules based on library names
and identifier keywords. This specialist learns the same rules from examples.

Training data: mine existing headers using the auto-tagger rules as labels.
Script to write: `scripts/gen_semantic_tag_data.py`

### 2.5 — `component_suffix` specialist (12-way)

Input: 1-sentence description of what a component does
Output: naming suffix that fits the role:
  Validator / Scorer / Auditor / Generator / Processor / Selector /
  Advisor / Binder / Extractor / Composer / Builder / Registry

Training data: 93 existing `editor/src/*.h` headers as labeled examples.
Each filename encodes the ground-truth suffix. Thin data but strong patterns.
Script to write: `scripts/gen_component_suffix_data.py`

---

## Phase 3 — Data Collection (Running in Background)

**Goal:** Make every LLM tool call a training data generator.

### 3.1 — Instrument bounded sub-decisions

Add post-processing to `tools/mcp/export_taskitem_run_for_lora.sh` to extract
per-decision labels from each pipeline run. When `generate_taskitems` emits a task
with `verificationType: "integration"`, append `(step_description, "integration")`
to `training_data/specialists/verification_type_examples.jsonl`.

### 3.2 — Annotation capture hook

When the LLM fills an `AutomatabilityAnnotation` or `IntentAnnotation` on any
AST node, capture `(input_features, annotation_value)` as a training example.
These accumulate automatically with normal usage.

### 3.3 — Blast radius instrumentation

When `BlastRadiusAnalyzer` runs, emit `(dependentCount, blastRadius, callerCount) → riskLevel`
as a labeled example. This grows the `change_risk_tier` training set with every
real analysis run.

---

## Phase 4 — Integration

**Goal:** Wire specialists into the MCP tool layer.

### 4.1 — Dispatch server

`scripts/dispatch.py` — loads all trained specialists at startup, accepts JSON,
returns predictions. All specialists run in a single batched forward pass.

### 4.2 — Pre-annotation pass

New MCP tool: `whetstone_annotate_nodes` — given a list of AST node descriptors,
runs the specialist fleet and returns predicted annotation values before the LLM
is called. The LLM then only handles open-text fields and `human`-tier decisions.

### 4.3 — Automatability routing

Once `automatability_strategy` specialist is trained and validated:
- `deterministic` nodes: handled by formula/rule engine, no inference
- `template` nodes: handled by template instantiation
- `slm` nodes: handled by specialist dispatch
- `llm` nodes: LLM call with pre-filled context from specialists
- `human` nodes: escalated via `ReviewAnnotation.required = true`

---

## Phase 5 — Code Generation Specialists

**Goal:** Decompose "generative" decisions into bounded sub-decisions.

Once Phase 2 establishes the full annotation vocabulary, code generation becomes:

```
Node with annotations
    → automatability_strategy = "template"         ← specialist decides
    → intent_category = "transformation"           ← specialist decides
    → structural_pattern = "pipeline"              ← specialist decides (new)
    → control_flow_pattern = "early-return"        ← specialist decides (new)
    → error_handling_style = "Result<T,E>"         ← specialist decides (new)
    → projection language = "cpp"                  ← human or router decides
    ↓
Deterministic template instantiation
    ↓
Code skeleton
    ↓
LLM fills only: identifier names, literal values, comments  ← narrow residual
```

Specialists for this phase:
- `structural_pattern` (factory/visitor/strategy/builder/observer/pipeline/scorer/registry...)
- `control_flow_pattern` (linear/if-else-chain/early-return/loop/recursive/dispatch-table...)
- `error_handling_style` (exception/Result/optional/expected/assert/error-code...)
- `data_structure_choice` (vector/map/deque/set/flat-array/ring-buffer...)

Each of these is a bounded classification over the AST node's role and annotation
context. Each can be trained from the existing `editor/src/` headers as examples.

---

## Tracking Progress

| Specialist | Phase | Status | Accuracy | Checkpoint |
|-----------|-------|--------|----------|------------|
| `verification_type` | 0 | ✅ DONE | 83.7% | `/mnt/storage/fabricate_runs/whetstone_verification_type/` |
| `confidence_tier` | 0 | ✅ DONE | 100% | `/mnt/storage/fabricate_runs/whetstone_confidence_tier/` |
| `prereq_op_selector` | 1.1 | ⬜ planned | — | — |
| `worker_type` | 1.2 | ⬜ planned | — | — |
| `gap_class` | 1.3 | ⬜ planned | — | — |
| `escalation_predictor` | 1.4 | ⬜ planned | — | — |
| `automatability_strategy` | 2.1 | ⬜ planned (6-way, needs Annotation.h update first) | — | — |
| `intent_category` | 2.2 | ⬜ planned | — | — |
| `change_risk_tier` | 2.3 | ⬜ planned | — | — |
| `semantic_tag` | 2.4 | ⬜ planned | — | — |
| `component_suffix` | 2.5 | ⬜ planned | — | — |
| `structural_pattern` | 5 | ⬜ planned | — | — |
| `control_flow_pattern` | 5 | ⬜ planned | — | — |
| `error_handling_style` | 5 | ⬜ planned | — | — |

---

## What the Next Agent Should Do First

1. **Read `README.md`** and this file completely
2. **Verify Phase 0 specialists** by running `infer.py --eval-tsv` on both held-out sets
3. **Start Phase 2.1** (`automatability_strategy`) — it is the most architecturally
   significant specialist and training data already exists in `editor/src/*.h`
4. **Then Phase 1.1** (`prereq_op_selector`) to complete the pipeline layer
5. **Do not train per-language specialist variants** — the AST is language-neutral;
   projection is handled downstream by deterministic grammar rules

---

## Connection to Other Projects

**WhetstoneAI_Fabricate** (`/home/bill/Documents/WhetstoneAI_Fabricate/`):
Training infrastructure. Binary, harness, eval tools. The RSA experiments
(see `HANDOFF-2026-03-28-RSA.md`) validated the specialist architecture at scale.
The label convention fix (`swap_label_rows.py`) is documented there — read it
before resuming any checkpoint with a changed `--labels` order.

**Tacit Codex** (`tacit_codex/`):
`AutomatabilityAnnotation.strategy` in `editor/src/ast/Annotation.h` is the
Tacit Codex entropy classification built into the AST schema. Training the
`automatability_strategy` specialist is the concrete implementation of the
Tacit Codex collapse-to-formal-structure program applied to individual AST nodes.
The entropy ledger in HiveMind tracks the same quantity at the job-scheduling layer.

**HiveMind** (`/home/bill/Documents/hivemind/`):
Future state: a `generate_taskitems` job becomes a fan-out of N specialist jobs
+ 1 LLM job (only for `llm`-tier nodes), merged by coordinator. The automatability
tier determines which job type each node gets dispatched to.