diff --git a/specialists/HANDOFF-2026-03-28.md b/specialists/HANDOFF-2026-03-28.md new file mode 100644 index 0000000..34712a1 --- /dev/null +++ b/specialists/HANDOFF-2026-03-28.md @@ -0,0 +1,239 @@ +# Whetstone Specialist Fleet — Handoff 2026-03-28 + +**Session date:** 2026-03-28 +**Status:** Phase 0 complete. Two specialists trained and validated. Clean stop. + +--- + +## What This Directory Is + +A fleet of ~800KB transformer specialists that replace bounded-output LLM decisions +in the Whetstone MCP pipeline. Each specialist handles one classification decision, +runs in under a millisecond, and is trained using the WhetstoneAI Fabricate binary +and harness unchanged. + +**Origin:** This grew directly from the WhetstoneAI Fabricate RSA experiments +(`/home/bill/Documents/WhetstoneAI_Fabricate/`). The same ~213K-param architecture +that reached 96.3% in-distribution accuracy on domain routing was applied to +Whetstone's bounded taskitem decisions. No new training infrastructure was written. + +**Full context:** Read `README.md` and `ROADMAP.md` in this directory before +starting work. They contain the architectural framing, the AST-first design +correction, the 6-tier automatability hierarchy, and the complete next-step plan. + +--- + +## What Was Built This Session + +### Two trained specialists + +**`verification_type`** — 5-way classifier +- Input: sprint step description text (e.g. `"whetstone_validate_schema MCP tool"`) +- Output: `unit / integration / schema / smoke / docs` +- Training: 978 examples mined from 260 sprint plan files via heuristic labeling +- Eval: 172 held-out examples +- Result: **83.7%** accuracy, grokked at step 1000, stopped at step 3000 +- Checkpoint: `/mnt/storage/fabricate_runs/whetstone_verification_type/` +- History: `specialists/runs/verification_type_history.json` + +**`confidence_tier`** — 3-way classifier +- Input: structural encoding `"conflicts=N ambiguity=M deps=D prereqs=P queueready=yes/no"` +- Output: `high / medium / low` +- Training: 1020 synthetic examples generated from the formula in `TaskitemConfidenceAmbiguity.h` +- Result: **100%** accuracy, grokked at step 1000, stopped at step 3000 +- Checkpoint: `/mnt/storage/fabricate_runs/whetstone_confidence_tier/` +- History: `specialists/runs/confidence_tier_history.json` + +### Scripts written + +``` +specialists/scripts/gen_verification_type_data.py mines sprint plans → TSV +specialists/scripts/gen_confidence_tier_data.py generates synthetic formula data → TSV +specialists/scripts/train_verification_type.sh wraps run_grokking_until.py +specialists/scripts/train_confidence_tier.sh wraps run_grokking_until.py +specialists/scripts/infer.py single/batch/eval-tsv/REPL inference +``` + +### Inference example + +```bash +# From whetstone_DSL root, using fabricate venv: +cd /home/bill/Documents/WhetstoneAI_Fabricate && \ +.venv/bin/python3 tools/eval_multiclass.py \ + --checkpoint /mnt/storage/fabricate_runs/whetstone_verification_type/checkpoint.bin \ + --db /mnt/storage/fabricate_runs/whetstone_verification_type/fabricate.db \ + --dataset /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/verification_type_eval.tsv \ + --labels "unit,integration,schema,smoke,docs" +# → overall: 144/172 = 83.7% +``` + +--- + +## Key Technical Facts + +**Training data format** (required by Fabricate binary): +``` +\t\t +``` +Label is an INTEGER index (0-based) matching the `--labels` flag order. +**String labels in column 1 will crash the binary.** The generators write integers. + +**`--labels` flag:** `"unit,integration,schema,smoke,docs"` means label 0=unit, +1=integration, 2=schema, 3=smoke, 4=docs. Order must match the TSV integers. + +**Model architecture** (all specialists use this): +`--hidden-dim 128 --layers 1 --heads 1 --seq-len 96 --batch-size 32` +Result: ~213K parameters, ~800KB checkpoint. + +**Grokking is normal.** Training logs show "Fresh start" (no prior checkpoint in +output dir), then accuracy jumps at step 1000. This is expected behavior. The +harness stops automatically 3 blocks after grokking. + +**Label/checkpoint alignment bug** (from RSA experiments — do not repeat): +If you resume a checkpoint with a changed `--labels` order, the token rows +misalign. Fix with `tools/swap_label_rows.py` in the Fabricate repo. +See `WhetstoneAI_Fabricate/HANDOFF-2026-03-28-RSA.md` for full history. + +--- + +## Important Architectural Decisions Made This Session + +### 1. AST-first, not tool-first + +Whetstone is an AST-as-source-of-truth system. C++, Python, etc. are projections. +Specialists operate at the AST level — one specialist per decision type covers all +languages. Do NOT train per-language variants of decisions that live in the AST +layer. The projection from AST to language is handled by deterministic grammar rules. + +### 2. The 6-tier automatability hierarchy + +The current `AutomatabilityAnnotation` in `editor/src/ast/Annotation.h` has 5 tiers. +It is missing the `specialist` tier. The correct hierarchy is: + +``` +deterministic → formula, zero parameters, always same output +template → deterministic structural composition +specialist → bounded output vocabulary, fuzzy, ~800KB ← THIS PROJECT +slm → open vocabulary generation, ~1-7GB +llm → full generative, ~14B+ +human → not yet formalizable +``` + +Boundary rule: if the output can be expressed as a fixed enum → specialist. +If it requires generating novel tokens → slm or llm. + +**`Annotation.h` needs a one-line update** to add `"specialist"` to the strategy +enum before Phase 2.1 training. This is called out in ROADMAP.md. + +### 3. Specialist drives template, not replaces it + +The specialist and template tiers work as a pipeline, not alternatives: +- Specialist picks which template + fills the bounded slot decisions +- Template deterministically expands those decisions into output +- LLM handles only the residual open-text portions + +### 4. The existing AST annotation schema IS the training target vocabulary + +`editor/src/ast/Annotation.h` already defines the output schema for most +specialists we need to train. Key fields: +- `AutomatabilityAnnotation.strategy` — the Tacit Codex entropy classifier on nodes +- `IntentAnnotation.category` — validation/transformation/io/coordination/computation/init +- `RiskAnnotation.dependentCount` — downstream caller count (input feature) +- `ComplexityAnnotation.cognitiveComplexity` — 1-10 scale (input feature) + +Specialists populate these fields automatically at near-zero cost, replacing +LLM annotation calls. + +--- + +## Exact Next Steps (Phase 1 + Phase 2.1) + +### Immediate: Phase 1 pipeline specialists + +Four specialists to train for the taskitem pipeline layer. All follow the same +pattern as `verification_type` and `confidence_tier`. + +**1.1 — `prereq_op_selector` (4-way)** +Write `scripts/gen_prereq_op_data.py` generating synthetic task descriptions for: +- `standard` (0): [validate-intake, resolve-dependencies] — default path +- `needs_review` (1): [validate-intake, architect-review] — breaking/cross-cutting changes +- `needs_approval` (2): [validate-intake, manual-approval] — experimental/risky tasks +- `full_gates` (3): [validate-intake, architect-review, manual-approval] — both signals +Train: `--labels "standard,needs_review,needs_approval,full_gates"` + +**1.2 — `worker_type` (4-way)** +Write `scripts/gen_worker_type_data.py` mining sprint plans with keyword heuristics: +- `implementer` (0): implement/add/build/write/create +- `reviewer` (1): review/audit/verify/check/validate +- `architect` (2): design/architect/plan/structure/refactor +- `qa` (3): test/coverage/regression/integration summary +Train: `--labels "implementer,reviewer,architect,qa"` + +**1.3 — `gap_class` (5-way)** +Write `scripts/gen_gap_class_data.py` with synthetic taskitem issue descriptions. +Classes: under_constrained / environment_gap / planning_gap / cross_project_gap / generator_capability_gap + +**1.4 — `escalation_predictor` (binary)** +Write `scripts/gen_escalation_data.py` from formula: `escalate = confidence < 60 OR ambiguity > 0`. +Expected 100% accuracy like `confidence_tier`. +Train: `--labels "proceed,escalate"` + +### After Phase 1: Phase 2.1 — `automatability_strategy` (6-way) + +**First:** Add `"specialist"` to strategy enum in `editor/src/ast/Annotation.h`. + +Write `scripts/gen_automatability_data.py` using the 93 existing `editor/src/*.h` +headers as labeled examples. Each header encodes the ground-truth tier: +- Pure formula classes → `deterministic` +- Schema-to-X converters → `template` +- Classifiers/scorers with bounded output → `specialist` +- Text generators with constrained vocabulary → `slm` +- Open-ended generators → `llm` + +Train: `--labels "deterministic,template,specialist,slm,llm,human"` + +This is the most architecturally significant specialist — once trained, every node +in the AST can be tagged with its automatability tier without an LLM call. + +--- + +## What Is NOT Done / Known Gaps + +- `infer.py` is a standalone inference tool but not yet wired into any MCP tool +- No dispatch server yet (`scripts/dispatch.py` — Phase 4.1 in roadmap) +- `Annotation.h` still has the old 5-tier enum (needs `"specialist"` added) +- The `verification_type` specialist has 16% error rate concentrated at the + schema/unit/integration boundary — the heuristic labeling is imperfect there. + Acceptable for now; fix by reviewing ~50 misclassified examples if accuracy matters. +- JSONL pipeline run records (18,519) are too homogeneous for text-based specialists. + Sprint plans are the right source. Do not try to use JSONL for text classifiers. + +--- + +## Connection to WhetstoneAI Fabricate + +The Fabricate binary and harness are consumed as-is. Do not modify them. + +``` +Binary: /home/bill/Documents/WhetstoneAI_Fabricate/build/whetstone_fabricate +Harness: /home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py +Eval: /home/bill/Documents/WhetstoneAI_Fabricate/tools/eval_multiclass.py +Infer: /home/bill/Documents/WhetstoneAI_Fabricate/tools/eval_with_vocab.py +``` + +The Fabricate venv at `/home/bill/Documents/WhetstoneAI_Fabricate/.venv` has all +required packages. Use `.venv/bin/python3` for all training and eval calls. + +--- + +## Connection to Tacit Codex + +The specialist fleet is the concrete implementation of the Tacit Codex program: +each trained specialist is a named, deployed piece of collapsed tacit knowledge. +The `AutomatabilityAnnotation.strategy` field, once populated by the Phase 2.1 +specialist, directly encodes the entropy tier of every AST node — which is the +per-node Tacit Codex entropy ledger entry. + +The same pattern (hard constraints + fuzzy contextual decisions) also applies to +WHIMP (interaction design) with the 722-metric DuckDB as the input feature vocabulary. +That connection is documented in the session conversation but not yet in any file. diff --git a/specialists/HANDOFF-2026-03-31.md b/specialists/HANDOFF-2026-03-31.md new file mode 100644 index 0000000..162c406 --- /dev/null +++ b/specialists/HANDOFF-2026-03-31.md @@ -0,0 +1,93 @@ +# Specialist Fleet Handoff — 2026-03-31 + +## Where We Are + +Switched training infrastructure from Fabricate (C++ binary + SQLite) to **pure PyTorch**. +Fabricate was causing uninterruptible D-state IO blocking because `/mnt/storage` is a spinning +HDD and Fabricate writes continuous SQLite WAL data (~100MB/min) even with morphisms disabled. + +## What Was Built This Session + +### New Files +- `specialists/scripts/train_specialist_pt.py` — PyTorch training script (replaces `run_grokking_until.py`) +- `specialists/eval/eval_specialist_pt.py` — PyTorch eval script (replaces `eval_specialist.py`) + +### Updated Files +- `specialists/scripts/capacity_sweep.py` — now calls `train_specialist_pt.py` + `eval_specialist_pt.py` +- `specialists/scripts/prereq_op_binary_split.py` — same, `evaluate_combined` rewritten in PyTorch +- Both scripts now use `checkpoint.pt` instead of `checkpoint.bin` + `fabricate.db` +- TIERS updated: `heads` field added (4/6/8 for small+/medium/large) — PyTorch supports multi-head unlike Fabricate + +### PyTorch training script details +- No SQLite, no WAL. Saves `checkpoint.pt` + `history.json` only. +- History JSON format compatible with `capacity_sweep.py`: `{"history": [{"step": N, "overall": float, "loss": float}]}` +- Same CLI interface as `run_grokking_until.py` for drop-in use by sweep scripts +- Grokking detection + early stop implemented (same logic: loss < threshold OR acc jump ≥ threshold) +- Checkpoint includes: model weights, optimizer state, vocab, config, labels, step count + +## Performance Issue — PARTIALLY FIXED + +**Root cause:** DataLoader restarts ~833×/1000 steps when dataset has 300 examples + batch_size=256. +Each restart has Python overhead → 38s per 1000-step block (expected ~4s). + +**Fix applied (untimed):** Load all training data onto GPU once as a tensor, sample via +`torch.randint` each step instead of DataLoader. This eliminates all Python/CPU overhead per step. + +```python +all_x = torch.stack([row[0] for row in train_ds]).to(device) +all_y = torch.tensor([row[1] for row in train_ds], ..., device=device) +# per step: +idx = torch.randint(0, n_train, (batch_size,), device=device) +x, y = all_x[idx], all_y[idx] +``` + +**This fix is in the code but was NOT timed before session ended.** + +## First Task Next Session + +1. Time the fixed training script: +```bash +cd /home/bill/Documents/CLionProjects/whetstone_DSL +time /home/bill/Documents/WhetstoneAI_Fabricate/.venv/bin/python3 \ + specialists/scripts/train_specialist_pt.py \ + --dataset specialists/data/generated/worker_type_train.tsv \ + --eval-dataset specialists/data/generated/worker_type_eval.tsv \ + --labels "implementer,reviewer,architect,qa" \ + --out-dir /tmp/pt_timing_test \ + --hidden-dim 256 --layers 2 --heads 4 \ + --lr 0.001 --weight-decay 0.01 \ + --max-steps 3000 --batch-size 256 \ + --stop-after-grokking-blocks 99 \ + --reset +``` + +Expected: ~4-8 seconds per 1000-step block (was 38s before fix), so ~12-25s for 3000 steps. +If correct, pilot sweep (all 4 gates × 3 tiers) should complete in ~5-10 minutes. + +2. If timing is good, run: +```bash +bash specialists/scripts/sweep_all_gates.sh --pilot-only +``` + +3. Analyze pilot results in `specialists/eval/results/summary.json`, decide which gates + need full training vs already good enough. + +## Current Baseline Accuracies (small Fabricate models, unchanged) +- `worker_type`: 88.5% (implementer residual-category issue, 64% implementer acc) +- `verification_type`: 84.9% (schema→integration confusion, capacity limited) +- `prereq_op`: 69.4% (standard residual-category, over-fires approval gate) +- `automatability`: 69.6% (deterministic 0% — pure residual-category collapse) + +Target: 98-99% for all gates. Larger PyTorch models (no multi-head constraint) should help. + +## Key Architecture Notes +- Fabricate constraint: `gpu_transformer` engine only supports `heads=1` +- PyTorch: any heads where `hidden_dim % heads == 0` — we now use 4/6/8 +- Run dirs are still on `/mnt/storage/fabricate_runs/` — this is fine, `.pt` files are + tiny (few MB) and written infrequently (once per 1000 steps), not continuously +- The old `whetstone_prereq_op_large` run dir on the HDD may still have a D-state + Fabricate process associated. Check `pgrep fabricate` at session start; kill if present. + +## Venv +All scripts use: `/home/bill/Documents/WhetstoneAI_Fabricate/.venv/bin/python3` +PyTorch 2.10+cu128 is installed there. CUDA is available (RTX 3060). diff --git a/specialists/README.md b/specialists/README.md new file mode 100644 index 0000000..2ec8b48 --- /dev/null +++ b/specialists/README.md @@ -0,0 +1,268 @@ +# Whetstone Specialist Fleet + +**What this is:** A system of ~800KB transformer specialists that replace bounded-output +LLM decisions inside the Whetstone MCP pipeline. Each specialist handles exactly one +classification or scoring decision, runs in under a millisecond, and can be held +resident in GPU VRAM alongside thousands of peers. + +This is not a LoRA. It is not a fine-tuned 14B model. It is a different architecture +for a different problem. + +--- + +## Origin: WhetstoneAI Fabricate Experiments + +This project grew out of the WhetstoneAI_Fabricate experiments at +`/home/bill/Documents/WhetstoneAI_Fabricate/`. + +Those experiments were training tiny transformer models (~213K parameters) to solve +structured reasoning problems — math ordering, logic gates, state machines, modular +arithmetic, DAG traversal. Each model was a binary classifier (true/false) over +reasoning chains. The key finding was the **Routed-Specialist Architecture (RSA)**: + +- A small router model (5-way classifier) reads a raw question and picks which + domain expert to consult +- The chosen specialist (binary classifier) answers the question +- End-to-end composed accuracy: 96.3% in-distribution, 76.7% OOD after hardening +- Total model size: router (~800KB) + 5 specialists (~800KB each) = ~5MB +- Inference: microseconds per query on GPU + +The critical realization: **the Whetstone taskitem pipeline is already doing this +same structure, but with a 14B LLM handling all the decisions.** Every tool call +involves the LLM choosing from bounded option sets — verification type, prerequisite +ops, confidence tier, worker role, quality score tier. These are exactly the +classifier-shaped problems that the Fabricate architecture solves at 800KB. + +The WhetstoneAI_Fabricate binary and training harness are reused here unchanged. +No new training infrastructure was written. + +--- + +## The AST Is Already Rich — Specialists Predict Its Fields + +Whetstone is not a C++ code generator. C++, Python, Rust, and all other output +languages are *projections* of an AST-as-source-of-truth. The AST uses +language-neutral `conceptType` names (`Function`, `ClassDeclaration`, `IfStatement`) +that map across 19+ languages through a grammar layer. This means specialists operate +at the AST level and cover all languages simultaneously — there are no per-language +variants for decisions that live in the AST. + +More importantly: the Whetstone AST already carries annotation metadata that most +ASTs never have. The annotation schema in `editor/src/ast/Annotation.h` (60+ types) +includes: + +``` +RiskAnnotation.dependentCount int how many callers/consumers downstream +CallHierarchyInfo.callerIds list which functions call this one +BlastRadiusReport.riskLevel enum none/low/medium/high (blast radius) +IntentAnnotation.category enum validation/transformation/io/... +ComplexityAnnotation.cognitiveComplexity int 1-10 scale +AutomatabilityAnnotation.strategy enum deterministic|template|specialist|slm|llm|human +AutomatabilityAnnotation.confidence float 0.0–1.0 +``` + +NOTE: Current `Annotation.h` is missing `specialist` in this enum. Add it before +training the `automatability_strategy` specialist. The 6 tiers in order: +formula → template → specialist (~800KB bounded classifier) → slm → llm → human. + +**`AutomatabilityAnnotation.strategy` is the Tacit Codex entropy classification +built directly into the AST schema.** It encodes whether each node's decisions +should be handled by a formula, template instantiation, a tiny specialist, +a small LLM, a large LLM, or a human. Training the `automatability_strategy` +specialist means populating this field automatically across the entire codebase — +every node gets its entropy tier assigned without an LLM call. + +`RiskAnnotation.dependentCount` is the "how many things downstream depend on this?" +metric the system already tracks. Combined with blast radius analysis +(`BlastRadiusAnalyzer.h`) and call hierarchy (`ContextAPI.h`), the AST already +measures change risk. These are input features for the `change_risk_tier` specialist. + +## The Core Premise + +### What an MCP tool call currently does + +When you call `whetstone_generate_taskitems`, the LLM receives context and makes +roughly these decisions: + +``` +How many milestones? → integer 1-6 +Verification type? → unit | integration | schema | smoke | docs +Prereq ops needed? → combo of 4 possible ops +Worker type? → implementer | reviewer | architect | qa +Confidence level? → integer 0-100 (computed from formula) +Should escalate? → bool +...and then the generative parts: +Task title text? → open text +Acceptance criteria text? → open text +``` + +The first group are **bounded decisions** — the output is always one choice from +a small fixed set. These are classifier problems. The second group are **generative +decisions** — the output is open text. These still require the LLM. + +### What the specialist fleet does instead + +Every bounded decision becomes a specialist call. All specialists fire in parallel +before or instead of the LLM call. The LLM, if called at all, only handles the +open-text portions — with the bounded decisions already pre-filled. + +``` +[specialist: verification_type] ──► unit/integration/schema/smoke/docs ~0.5ms +[specialist: prereq_op_selector] ──► [validate-intake, resolve-deps] ~0.5ms +[specialist: confidence_tier] ──► high/medium/low ~0.5ms +[specialist: worker_type] ──► implementer ~0.5ms + ↓ all parallel, all in <2ms total +[LLM: generate task title] ──► "ResourceLocks schema extension" ~3s +[LLM: write acceptance criteria] ──► "..." ~2s +``` + +### Why this scales + +An RTX 3060 has 12GB VRAM. One quantized 14B LLM takes ~8GB. One specialist takes +~800KB. The same card that holds one LLM can hold **~14,000 specialists resident +simultaneously**. + +50 bounded decision types × 20 domain variants each = 1,000 specialists = 800MB VRAM. +All 1,000 can run in a single batched forward pass. + +--- + +## The Philosophical Anchor (Tacit Codex) + +This project is an instance of the broader +[Tacit Codex](../tacit_codex/) work. + +The central claim: **"LLM required" is not a permanent category. It is a label for +decisions we haven't formalized yet.** + +Every decision in software generation is either: +1. **Deterministic** — given the same inputs, always the same output (e.g., filename + = `FeatureName.h`, MCP tool name = `whetstone_` + snake_case verb + noun) +2. **Statistical** — the output is one of N known options, predictable from features + (e.g., verification type from step description) → specialist +3. **"Human mind magic"** — the residual category for decisions that feel non-mechanical + +The third category is not permanent. It marks where the decision space hasn't been +decomposed fine enough yet. "Write a function body" feels like magic until you +decompose it into: structural pattern (specialist) × error handling style (specialist) +× data structure choice (specialist) × loop form (specialist) × variable naming +pattern (specialist), all composed through deterministic template instantiation. + +**The LLM is the scaffolding.** It operates at high entropy while the specialists +are being built. Every LLM decision is an implicit training example for the specialist +that will eventually replace it. The Tacit Codex entropy ledger tracks which decisions +are still unformalized — the specialist fleet is what the collapsed version looks like. + +--- + +## Current State (as of 2026-03-28) + +Two specialists trained and validated: + +| Specialist | Classes | Accuracy | Training data | Checkpoint | +|-----------|---------|----------|---------------|------------| +| `verification_type` | unit/integration/schema/smoke/docs (5-way) | **83.7%** | 978 examples from 260 sprint plans | `/mnt/storage/fabricate_runs/whetstone_verification_type/` | +| `confidence_tier` | high/medium/low (3-way) | **100%** | 1020 synthetic examples from formula | `/mnt/storage/fabricate_runs/whetstone_confidence_tier/` | + +The `verification_type` specialist was trained entirely from the sprint plan files +that already exist in this repo — no external data, no hand-labeling. The sprint +step descriptions (`### Step NNN: `) are labeled by heuristic rules +that classify each description into its verification type. + +The `confidence_tier` specialist was trained on synthetic data generated from the +actual formula in `TaskitemConfidenceAmbiguity.h`. It learned the formula exactly. + +--- + +## Quick Start + +```bash +# From whetstone_DSL root + +# Generate training data (sprint plans → TSV) +python3 specialists/scripts/gen_verification_type_data.py --sprint-dir . + +# Train +bash specialists/scripts/train_verification_type.sh + +# Infer on a single example +python3 specialists/scripts/infer.py \ + --run-dir /mnt/storage/fabricate_runs/whetstone_verification_type \ + --labels "unit,integration,schema,smoke,docs" \ + --text "whetstone_detect_conflicts MCP tool" +# → integration (conf=1.00) + +# Interactive REPL (omit --text) +python3 specialists/scripts/infer.py \ + --run-dir /mnt/storage/fabricate_runs/whetstone_verification_type \ + --labels "unit,integration,schema,smoke,docs" + +# Evaluate accuracy on a held-out TSV +python3 specialists/scripts/infer.py \ + --run-dir /mnt/storage/fabricate_runs/whetstone_verification_type \ + --labels "unit,integration,schema,smoke,docs" \ + --eval-tsv specialists/data/generated/verification_type_eval.tsv +``` + +--- + +## Technical Notes for Next Agent + +**Training data format** (TSV, required by the Fabricate binary): +``` +\t\t +``` +- `integer_label`: index into `--labels` list (0-based) +- `hops`: always `0` for Whetstone specialists (unused, required by format) +- `text`: the input text to classify + +**The `--labels` flag** maps integer indices to token strings the model emits. +`--labels "unit,integration,schema,smoke,docs"` means label 0 = "unit", 1 = "integration", etc. +**The label column must be an integer, not a string.** String labels cause a crash. + +**Fabricate binary**: `/home/bill/Documents/WhetstoneAI_Fabricate/build/whetstone_fabricate` +**Training harness**: `/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py` +**Eval tool**: `/home/bill/Documents/WhetstoneAI_Fabricate/tools/eval_multiclass.py` +**Inference helpers**: `/home/bill/Documents/WhetstoneAI_Fabricate/tools/eval_with_vocab.py` + +**Model architecture** (fixed for all specialists): +`--hidden-dim 128 --layers 1 --heads 1 --seq-len 96 --batch-size 32` +This produces ~213K parameters, ~800KB checkpoint. + +**Grokking** is the normal training outcome. The model memorizes first (high training +accuracy, low eval accuracy), then at some step (usually 1000-3000) the eval accuracy +jumps sharply. The harness detects this and stops automatically. Do not be alarmed by +"Fresh start" at the top of training logs — this means no prior checkpoint exists in +the output dir, not that the weights are random (they're initialized from scratch as +intended). + +**Known issue from RSA work**: If you ever resume training a specialist from an +existing checkpoint while changing the `--labels` order, the token rows in the +checkpoint will be misaligned. Use +`/home/bill/Documents/WhetstoneAI_Fabricate/tools/swap_label_rows.py` to fix this +before resuming. See `WhetstoneAI_Fabricate/HANDOFF-2026-03-28-RSA.md` for the +full story of this bug. + +--- + +## Files in This Directory + +``` +specialists/ + README.md ← this file + ROADMAP.md ← phased implementation plan + scripts/ + gen_verification_type_data.py extract sprint plans → labeled TSV + gen_confidence_tier_data.py generate synthetic formula data → TSV + train_verification_type.sh train verification_type specialist + train_confidence_tier.sh train confidence_tier specialist + infer.py single/batch/eval/REPL inference + data/ + generated/ TSVs produced by gen_* scripts (committed) + mined/ data extracted from JSONL pipeline runs (future) + docs/ + ARCHITECTURE.md full decision-point inventory + design rationale + runs/ + *_history.json grokking history per specialist (committed) + *_training.log full training logs (gitignore these) +``` diff --git a/specialists/ROADMAP.md b/specialists/ROADMAP.md new file mode 100644 index 0000000..2cad5e6 --- /dev/null +++ b/specialists/ROADMAP.md @@ -0,0 +1,388 @@ +# Whetstone Specialist Fleet — Roadmap + +**Read README.md first.** This document assumes you understand the core premise. + +**Guiding principle:** "LLM required" is not a permanent category. Every decision is +either deterministic, statistical, or not yet decomposed. This roadmap is the plan +for collapsing the "not yet decomposed" pile, one specialist at a time. + +**Architectural correction (important):** Whetstone is NOT a C++ code generator. +C++, Python, Rust, and all other output languages are *projections* of an +AST-as-source-of-truth. The AST uses language-neutral `conceptType` names +(`Function`, `ClassDeclaration`, `IfStatement`, etc.) that map uniformly across +19+ target languages. Specialists operate at the AST level — one specialist per +decision type covers all projection languages. Do not train per-language variants +of decisions that live in the AST layer. + +--- + +## The AST Is Already Rich + +Whetstone's AST is not a minimal syntax tree. It carries annotation metadata that +most derived ASTs never have. Key files: + +- `editor/src/ast/ASTNode.h` — base node with `conceptType`, `semanticHash`, role-based children +- `editor/src/ast/Annotation.h` — 60+ annotation types (see Subject breakdown below) +- `editor/src/SemanticCoreIR.h` — cross-language IR with `intentTags` +- `editor/src/SemanticTags.h` — 10-tag vocabulary (`@serialize`, `@crypto`, `@io`, etc.) +- `editor/src/ContextAPI.h` — `CallHierarchyInfo` with `callerIds`/`calleeIds` +- `editor/src/graduation/BlastRadiusAnalyzer.h` — impact scoring +- `editor/src/DependencyGraph.h` — dependency edges and critical path + +### The metadata that matters for specialists + +**Downstream impact** — already computed, available as input features: +``` +RiskAnnotation.dependentCount int how many callers/consumers +CallHierarchyInfo.callerIds list which functions call this one +BlastRadiusReport.riskLevel enum none/low/medium/high +BlastRadiusEntry.impactScore int numerical impact +FailureTriageScore.blastRadius int propagation count +``` + +**Intent and complexity** — already stored on nodes: +``` +IntentAnnotation.category enum validation/transformation/io/ + coordination/computation/initialization +IntentAnnotation.summary text 1-sentence description +ComplexityAnnotation.timeComplexity O(1)/O(n)/O(n^2)/... +ComplexityAnnotation.cognitiveComplexity 1-10 scale +SemanticTagAnnotation.tags list @serialize/@crypto/@io/@math/etc. +``` + +**The Tacit Codex annotation — already in the schema:** +``` +AutomatabilityAnnotation.strategy enum deterministic | template | slm | llm | human +AutomatabilityAnnotation.confidence float 0.0–1.0 +``` + +This annotation encodes the entire Tacit Codex hierarchy on individual AST nodes. +A specialist that predicts `automatability.strategy` for a given node *is* the +entropy classifier the Tacit Codex is building toward — it just needs to be trained. + +--- + +## Phase 0 — Foundation (COMPLETE) + +**Done:** +- `verification_type` specialist: 5-way, 83.7% accuracy + - Input: sprint step description text + - Output: unit / integration / schema / smoke / docs + - Checkpoint: `/mnt/storage/fabricate_runs/whetstone_verification_type/` +- `confidence_tier` specialist: 3-way, 100% accuracy + - Input: `"conflicts=N ambiguity=M deps=D prereqs=P queueready=yes/no"` + - Output: high / medium / low + - Checkpoint: `/mnt/storage/fabricate_runs/whetstone_confidence_tier/` +- Inference wrapper: `scripts/infer.py` + +**Key finding from Phase 0:** Sprint plan step descriptions are the right training +data source for text-based specialists. The 18,519 JSONL pipeline run records are +too homogeneous (2 task patterns repeated). Do not use JSONL as the text source. + +--- + +## Phase 1 — Pipeline Specialists (NEXT) + +**Goal:** Cover the remaining bounded decisions in `generate_taskitems` +and `validate_taskitem`. These are at the taskitem planning layer, above the AST. + +### 1.1 — `prereq_op_selector` (4-way) + +Input: task step description text +Output: +``` +0: standard → [validate-intake, resolve-dependencies] +1: needs_review → [validate-intake, architect-review] +2: needs_approval → [validate-intake, manual-approval] +3: full_gates → [validate-intake, architect-review, manual-approval] +``` +Training data: synthetic — generate task descriptions that signal each gate type. +Script to write: `scripts/gen_prereq_op_data.py` + +### 1.2 — `worker_type` (4-way) + +Input: task step description text +Output: implementer / reviewer / architect / qa +Training data: sprint plan files (same heuristic approach as `verification_type`). +Script to write: `scripts/gen_worker_type_data.py` + +### 1.3 — `gap_class` (5-way) + +Input: taskitem issue description text +Output: under_constrained / environment_gap / planning_gap / cross_project_gap / generator_capability_gap +Training data: synthetic — generate issue descriptions encoding each gap type. +Script to write: `scripts/gen_gap_class_data.py` + +### 1.4 — `escalation_predictor` (binary) + +Input: `"confidence=N ambiguity=M"` structural encoding +Output: escalate / proceed +Training data: synthetic from formula (`escalate = confidence < 60 OR ambiguity > 0`). +Expected accuracy: 100% (deterministic formula, same as `confidence_tier`). +Script to write: `scripts/gen_escalation_data.py` + +--- + +## Phase 2 — AST Annotation Specialists + +**Goal:** Train specialists that predict annotation fields on AST nodes. +These operate one level deeper than the taskitem pipeline — they answer questions +about individual AST nodes, not about sprint planning. + +The annotation system already defines the output schema. Specialists populate +annotation fields that are currently filled by the LLM or left empty. + +### 2.1 — `automatability_strategy` specialist (THE KEY ONE) — (6-way) + +**This is the Tacit Codex entropy classifier applied to AST nodes.** + +**IMPORTANT — update `editor/src/ast/Annotation.h` first.** The current enum +(`deterministic|template|slm|llm|human`) is missing the `specialist` tier. +The correct 6-tier hierarchy is: + +``` +deterministic same input → same output always. Formula. Zero parameters. +template deterministic composition of known structural patterns. No ambiguity. +specialist bounded output vocabulary (fixed N classes). Fuzzy/probabilistic. + ~800KB model, microseconds — THIS PROJECT's outputs. +slm open vocabulary, arbitrary token generation. ~1-7GB. +llm full generative, cross-context, novel reasoning. ~14B+. +human not yet formalizable. +``` + +Boundary between `specialist` and `slm`: can the output be a fixed enum? +Yes → specialist. Requires generating novel tokens → slm or llm. +Boundary between `template` and `specialist`: deterministic? Yes → template. +Probabilistic with a confidence score → specialist. + +Add `"specialist"` to the strategy field in `Annotation.h` before training. + +Input features (all from existing AST/annotation metadata): +``` +conceptType (Function/Class/Variable/EnumDeclaration/...) +intentCategory (validation/transformation/io/coordination/computation/init) +dependentCount (0-100+ from RiskAnnotation) +semanticTags (space-separated: @io @serialize @math...) +cognitiveComplexity (1-10 from ComplexityAnnotation) +hasBody (yes/no — skeleton vs implemented) +``` +Encoded as: `"type=Function intent=validation deps=12 tags=@io @serialize complexity=4 body=yes"` + +Output: deterministic / template / **specialist** / slm / llm / human + +**Why this is the most important specialist:** +Once trained, every node in the AST can be tagged with its automatability tier. +The pipeline routes each node's decisions to the appropriate handler: +- `deterministic` → formula/rule, no inference at all +- `template` → template instantiation, near-zero cost +- `specialist` → specialist fleet dispatch (~0.5ms) +- `slm` → small model call (~500ms) +- `llm` → full LLM call (~2-5s) +- `human` → escalate via ReviewAnnotation + +This replaces the current situation where every decision defaults to "llm" regardless. + +Training data: The 93 existing `editor/src/*.h` headers are labeled examples. +Each header class can be manually categorized. Starting corpus: ~93 examples +(thin but the patterns are strong — expect grokking with heavy oversampling). +Future: every new header added to the codebase extends the training set. + +Script to write: `scripts/gen_automatability_data.py` + +### 2.2 — `intent_category` specialist (6-way) + +Input: node name + semantic tags + parent conceptType (all language-neutral) +Output: validation / transformation / io / coordination / computation / initialization + +Training data: mine existing headers. The class name + role suffix strongly predicts +the category: +- `*Validator`, `*Scorer`, `*Auditor` → validation +- `*Generator`, `*Builder`, `*Composer` → transformation +- `*Loader`, `*Writer`, `*Exporter` → io +- `*Dispatcher`, `*Router`, `*Registry` → coordination +- `*Calculator`, `*Analyzer`, `*Solver` → computation +- `*Initializer`, `*Factory`, `*Bootstrap` → initialization + +This is trainable from class name patterns alone. 93 headers = 93 examples to start. +Script to write: `scripts/gen_intent_category_data.py` + +### 2.3 — `change_risk_tier` specialist (4-way) + +Input: `"deps=N callers=M impactScore=K blastRadius=R"` (all from existing metadata) +Output: low / medium / high / critical + +This specialist uses the blast radius and downstream dependency counts as its +primary input signal. A function with `deps=47` and `blastRadius=12` is high risk. +A function with `deps=0` and `callers=1` is low risk. + +Training data: synthetic from rules: +- critical: deps > 20 OR callerIds.count > 15 +- high: deps 10-20 OR callers 8-15 +- medium: deps 3-10 OR callers 2-8 +- low: deps 0-3, callers 0-2 + +Script to write: `scripts/gen_change_risk_data.py` +Expected accuracy: ~100% (formula-based, same as `confidence_tier`). + +### 2.4 — `semantic_tag` specialist (multi-label → primary tag, 10-way) + +Input: node name text (language-neutral conceptual name) +Output: primary semantic tag from vocabulary: + @serialize / @crypto / @io / @network / @math / @collection / + @concurrency / @test / @ui / @parse + +The `SemanticTags.h` file already has auto-tagging rules based on library names +and identifier keywords. This specialist learns the same rules from examples. + +Training data: mine existing headers using the auto-tagger rules as labels. +Script to write: `scripts/gen_semantic_tag_data.py` + +### 2.5 — `component_suffix` specialist (12-way) + +Input: 1-sentence description of what a component does +Output: naming suffix that fits the role: + Validator / Scorer / Auditor / Generator / Processor / Selector / + Advisor / Binder / Extractor / Composer / Builder / Registry + +Training data: 93 existing `editor/src/*.h` headers as labeled examples. +Each filename encodes the ground-truth suffix. Thin data but strong patterns. +Script to write: `scripts/gen_component_suffix_data.py` + +--- + +## Phase 3 — Data Collection (Running in Background) + +**Goal:** Make every LLM tool call a training data generator. + +### 3.1 — Instrument bounded sub-decisions + +Add post-processing to `tools/mcp/export_taskitem_run_for_lora.sh` to extract +per-decision labels from each pipeline run. When `generate_taskitems` emits a task +with `verificationType: "integration"`, append `(step_description, "integration")` +to `training_data/specialists/verification_type_examples.jsonl`. + +### 3.2 — Annotation capture hook + +When the LLM fills an `AutomatabilityAnnotation` or `IntentAnnotation` on any +AST node, capture `(input_features, annotation_value)` as a training example. +These accumulate automatically with normal usage. + +### 3.3 — Blast radius instrumentation + +When `BlastRadiusAnalyzer` runs, emit `(dependentCount, blastRadius, callerCount) → riskLevel` +as a labeled example. This grows the `change_risk_tier` training set with every +real analysis run. + +--- + +## Phase 4 — Integration + +**Goal:** Wire specialists into the MCP tool layer. + +### 4.1 — Dispatch server + +`scripts/dispatch.py` — loads all trained specialists at startup, accepts JSON, +returns predictions. All specialists run in a single batched forward pass. + +### 4.2 — Pre-annotation pass + +New MCP tool: `whetstone_annotate_nodes` — given a list of AST node descriptors, +runs the specialist fleet and returns predicted annotation values before the LLM +is called. The LLM then only handles open-text fields and `human`-tier decisions. + +### 4.3 — Automatability routing + +Once `automatability_strategy` specialist is trained and validated: +- `deterministic` nodes: handled by formula/rule engine, no inference +- `template` nodes: handled by template instantiation +- `slm` nodes: handled by specialist dispatch +- `llm` nodes: LLM call with pre-filled context from specialists +- `human` nodes: escalated via `ReviewAnnotation.required = true` + +--- + +## Phase 5 — Code Generation Specialists + +**Goal:** Decompose "generative" decisions into bounded sub-decisions. + +Once Phase 2 establishes the full annotation vocabulary, code generation becomes: + +``` +Node with annotations + → automatability_strategy = "template" ← specialist decides + → intent_category = "transformation" ← specialist decides + → structural_pattern = "pipeline" ← specialist decides (new) + → control_flow_pattern = "early-return" ← specialist decides (new) + → error_handling_style = "Result" ← specialist decides (new) + → projection language = "cpp" ← human or router decides + ↓ +Deterministic template instantiation + ↓ +Code skeleton + ↓ +LLM fills only: identifier names, literal values, comments ← narrow residual +``` + +Specialists for this phase: +- `structural_pattern` (factory/visitor/strategy/builder/observer/pipeline/scorer/registry...) +- `control_flow_pattern` (linear/if-else-chain/early-return/loop/recursive/dispatch-table...) +- `error_handling_style` (exception/Result/optional/expected/assert/error-code...) +- `data_structure_choice` (vector/map/deque/set/flat-array/ring-buffer...) + +Each of these is a bounded classification over the AST node's role and annotation +context. Each can be trained from the existing `editor/src/` headers as examples. + +--- + +## Tracking Progress + +| Specialist | Phase | Status | Accuracy | Checkpoint | +|-----------|-------|--------|----------|------------| +| `verification_type` | 0 | ✅ DONE | 83.7% | `/mnt/storage/fabricate_runs/whetstone_verification_type/` | +| `confidence_tier` | 0 | ✅ DONE | 100% | `/mnt/storage/fabricate_runs/whetstone_confidence_tier/` | +| `prereq_op_selector` | 1.1 | ⬜ planned | — | — | +| `worker_type` | 1.2 | ⬜ planned | — | — | +| `gap_class` | 1.3 | ⬜ planned | — | — | +| `escalation_predictor` | 1.4 | ⬜ planned | — | — | +| `automatability_strategy` | 2.1 | ⬜ planned (6-way, needs Annotation.h update first) | — | — | +| `intent_category` | 2.2 | ⬜ planned | — | — | +| `change_risk_tier` | 2.3 | ⬜ planned | — | — | +| `semantic_tag` | 2.4 | ⬜ planned | — | — | +| `component_suffix` | 2.5 | ⬜ planned | — | — | +| `structural_pattern` | 5 | ⬜ planned | — | — | +| `control_flow_pattern` | 5 | ⬜ planned | — | — | +| `error_handling_style` | 5 | ⬜ planned | — | — | + +--- + +## What the Next Agent Should Do First + +1. **Read `README.md`** and this file completely +2. **Verify Phase 0 specialists** by running `infer.py --eval-tsv` on both held-out sets +3. **Start Phase 2.1** (`automatability_strategy`) — it is the most architecturally + significant specialist and training data already exists in `editor/src/*.h` +4. **Then Phase 1.1** (`prereq_op_selector`) to complete the pipeline layer +5. **Do not train per-language specialist variants** — the AST is language-neutral; + projection is handled downstream by deterministic grammar rules + +--- + +## Connection to Other Projects + +**WhetstoneAI_Fabricate** (`/home/bill/Documents/WhetstoneAI_Fabricate/`): +Training infrastructure. Binary, harness, eval tools. The RSA experiments +(see `HANDOFF-2026-03-28-RSA.md`) validated the specialist architecture at scale. +The label convention fix (`swap_label_rows.py`) is documented there — read it +before resuming any checkpoint with a changed `--labels` order. + +**Tacit Codex** (`tacit_codex/`): +`AutomatabilityAnnotation.strategy` in `editor/src/ast/Annotation.h` is the +Tacit Codex entropy classification built into the AST schema. Training the +`automatability_strategy` specialist is the concrete implementation of the +Tacit Codex collapse-to-formal-structure program applied to individual AST nodes. +The entropy ledger in HiveMind tracks the same quantity at the job-scheduling layer. + +**HiveMind** (`/home/bill/Documents/hivemind/`): +Future state: a `generate_taskitems` job becomes a fan-out of N specialist jobs ++ 1 LLM job (only for `llm`-tier nodes), merged by coordinator. The automatability +tier determines which job type each node gets dispatched to. diff --git a/specialists/data/combined/prereq_op_eval.tsv b/specialists/data/combined/prereq_op_eval.tsv new file mode 100644 index 0000000..8c76cff --- /dev/null +++ b/specialists/data/combined/prereq_op_eval.tsv @@ -0,0 +1,36 @@ +2 0 `whetstone_enqueue_pair_upgrade` MCP tool +3 0 Deprecation/succession lifecycle framework +3 0 Security scan orchestrator for generated targets +1 0 Phase 12c Integration Tests +2 0 Reviewer quorum policy model for critical approvals +1 0 Phase 38a Integration +1 0 Phase 23b Integration + Sprint Summary +0 0 Reprojection safety policy +2 0 Stable corpus certification suite +0 0 Timing regression runner integration +0 0 `whetstone_get_tool_permissions` MCP tool +3 0 Cross-language rollout state snapshot model +1 0 Compatibility policy profiles by interface criticality +3 0 `whetstone_publish_roadmap_epoch` MCP tool +2 0 Security Report Generation +2 0 Compatibility/deprecation report artifact +1 0 Human Review Interface via MCP +2 0 HiveMind job publisher from editor +1 0 Parser/AST adapter interface schema +0 0 Sprint 76 integration summary + regression +1 0 Scenario — Multi-Model Orchestration +0 0 Autonomous loop controller with “green or explicit blocked” contract +2 0 Security-Aware Routing +0 0 Verifier trust weighting policy engine +0 0 Define verification and readiness surface | Build distributed rate limiter with fixed/sliding window modes and policy APIs. +1 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +1 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +2 0 Plan Replace search ranking pipeline behind feature flag with | Replace search ranking pipeline behind feature flag with staged rollout, telemetry checks, and automatic abort trigger. +0 0 Define verification and readiness surface | Build a JWT auth service with register, login, refresh token rotation, and password hashing. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua diff --git a/specialists/data/combined/prereq_op_train.tsv b/specialists/data/combined/prereq_op_train.tsv new file mode 100644 index 0000000..83505ce --- /dev/null +++ b/specialists/data/combined/prereq_op_train.tsv @@ -0,0 +1,204 @@ +2 0 `whetstone_get_constructive_rollout_status` runtime implementation +1 0 Target projection blocker taxonomy +2 0 Strict gate mode in production loop +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +3 0 Security scan orchestrator for generated targets +1 0 Multi-language constructive loop report artifact +2 0 `whetstone_get_constructive_rollout_status` MCP tool +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +2 0 Deprecation policy model for unstable/obsolete pairs +0 0 Projection schema extension for run catalogs +0 0 Panel Registration System +0 0 Sprint 146 integration summary + regression +3 0 `whetstone_publish_roadmap_epoch` MCP tool +1 0 Distributed path in FeedbackLoopOrchestrator +1 0 Continuity drill framework and scoring +3 0 Cross-language release evidence aggregator model +0 0 Scheme lowering/raising adapters +3 0 Cross-language release evidence aggregator model +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +1 0 Phase 13a Integration — Layout Persistence +2 0 Security Testing Skeleton Generation +0 0 Modifier-Edge Shortcut Notation +1 0 Architecture Templates +3 0 Deprecation/succession lifecycle framework +1 0 Language-specific method body scaffolding for core patterns +1 0 Initializer Lists + STL Patterns +3 0 Security scan orchestrator for generated targets +0 0 `whetstone_dry_run_patch` MCP tool +3 0 Cross-language rollout state snapshot model +3 0 Security scan orchestrator for generated targets +0 0 Cost telemetry integration in metrics stack +2 0 Constitutional invariants compliance checker +0 0 Property-based equivalence runner +2 0 Regional rollout profile integration +0 0 Compile/test proof binding +3 0 Cross-language release evidence aggregator model +2 0 `whetstone_get_certification_status` MCP tool +3 0 Deprecation/succession lifecycle framework +0 0 Sprint 193 Integration Summary +1 0 Multi-language replay corpus runtime index model +2 0 Benchmark suite for production capability claim +3 0 `whetstone_publish_roadmap_epoch` MCP tool +0 0 Build output parser — GCC/Clang format +2 0 Rollout stage model (`pilot`, `canary`, `broad`, `default`) +2 0 `whetstone_set_language_rollout_tier` MCP tool +0 0 Breadcrumb Navigation +1 0 Runtime-aware lowering extension points +1 0 Phase 15b Integration — Full Protocol Test +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +0 0 Python lowering adapter v2 (dynamic shape packets) +0 0 Canonical fixture and oracle dataset packs +0 0 Project context auto-injection +0 0 LTS baseline criteria schema +3 0 Security scan orchestrator for generated targets +0 0 `whetstone_validate_tool_contracts` MCP tool +1 0 Phase 20a Integration +3 0 Cross-language release evidence aggregator model +2 0 `whetstone_publish_next_block_plan` MCP tool +2 0 Staged cutover planner (`shadow`, `dual-run`, `cutover`) +3 0 Security scan orchestrator for generated targets +3 0 Security scan orchestrator for generated targets +0 0 Locals + Watches Panel Data Model +0 0 Context Bundle Minimizer +2 0 `whetstone_get_compliance_evidence` MCP tool +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +2 0 Security Annotation Types — Subject 10 +0 0 Benchmark scorecard generator per pair +1 0 Runtime-specific verification plugin interface +2 0 Certification job scheduler +3 0 Deprecation/succession lifecycle framework +2 0 Tiered rollout and rollback control model +2 0 Sprint 60 integration summary + release marker +3 0 Cross-language rollout state snapshot model +0 0 Constraint Violation Diagnostics +1 0 Phase 12a Integration Tests +3 0 Cross-language rollout state snapshot model +1 0 Deterministic blocker taxonomy extension +0 0 `whetstone_get_regeneration_decisions` MCP tool +1 0 Long-horizon roadmap planning engine v2 +0 0 Intake extraction confidence scoring +1 0 Phase 18b Integration + Sprint Summary +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +2 0 Certification badges for roles/workflows +0 0 Closeout consistency gate +3 0 Cross-language rollout state snapshot model +1 0 Binary/interface diff engine integration +2 0 Release Readiness Gate Pack +0 0 Source-vs-runtime drift comparator model +1 0 Phase 12d Integration + Sprint 12 Summary +1 0 `whetstone_verify_architecture_consistency` MCP tool +0 0 API/ABI preservation report artifact +0 0 Sprint 164 Integration Summary +3 0 `whetstone_publish_roadmap_epoch` MCP tool +0 0 `whetstone_capture_mcp_execution_trace` MCP tool +0 0 Template recommendation model by project fingerprint +0 0 Kotlin Generator +3 0 Deprecation/succession lifecycle framework +1 0 Cross-language capability contract normalizer model +2 0 Release certification packet model +2 0 Rollout blast-radius estimator +3 0 Cross-language release evidence aggregator model +2 0 Tenant-scoped waiver/approval ledger +1 0 Governance model refresh framework +2 0 `whetstone_publish_next_epoch_plan` MCP tool +3 0 Cross-language release evidence aggregator model +0 0 Embedded migration risk report template +1 0 Cross-language node identity preservation policy model +0 0 Key Symbol Rendering +1 0 Burn-down planning optimizer +3 0 Deprecation/succession lifecycle framework +2 0 Interop certification policy bindings +3 0 Cross-language release evidence aggregator model +3 0 Deprecation/succession lifecycle framework +2 0 Production summary hardening +3 0 `whetstone_publish_roadmap_epoch` MCP tool +2 0 Safety-critical deployment dossier artifact +3 0 `whetstone_publish_roadmap_epoch` MCP tool +2 0 Security-Preserving Translation +1 0 Swarm maintenance job decomposition model +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +1 0 Polyglot project migration graph model +1 0 Phase 10d Integration Tests +3 0 Cross-language rollout state snapshot model +1 0 Phase 18a Integration +2 0 Runtime-backed constructive release certification packet model +1 0 Phase 29a Integration +2 0 Production loop contract preflight +1 0 Phase 20b Integration + Sprint Summary +2 0 Pair-upgrade queue model +2 0 `whetstone_get_tool_deprecations` MCP tool +0 0 Wire `ArchitectIntakeProcessor` into `whetstone_architect_intake` handler +2 0 Contract drift classifier (`compatible`, `warning`, `breaking`) +1 0 Phase 23c Integration (tests: integration, regression) +1 0 `FeedbackLoopOrchestrator` core loop +1 0 WAT Cross-Language Projection +0 0 Sprint 187 Integration Summary +0 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Define verification and readiness surface | Build chat bot backend with slash commands, permission checks, and scheduled digests. +1 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +2 0 Define verification and readiness surface | Build a CMS backend with posts, tags, draft/publish workflow, and role-based access control. +0 0 Define verification and readiness surface | Build a multi-tenant limiter with token bucket + sliding window hybrid policy, burst fairness guarantees, and lock-free +1 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +2 0 Define verification and readiness surface | Build a CLI password vault with encrypted local storage and master-key unlock. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Define verification and readiness surface | Build a release notes generator from commits, PR labels, and issue references. +2 0 Define verification and readiness surface | Build a secrets rotation scheduler with lease tracking, retry plans, and notifications. +1 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +2 0 Define verification and readiness surface | Build a model registry API supporting stages, approvals, rollback, and signed artifact links. +1 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +1 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +1 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +1 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +0 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Define verification and readiness surface | Build inventory endpoints with stock reservation, release, and low-stock alerts. +0 0 Define verification and readiness surface | Build form builder backend with schema validation, submissions, and analytics counters. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +1 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +0 0 Define verification and readiness surface | Build experiment tracking backend storing params, metrics, artifacts, and compare queries. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +0 0 Define verification and readiness surface | Build a .env manager CLI with profile switch, validation, and secure redaction in output. +0 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Define verification and readiness surface | Build a secret rotation controller with staged rollout, canary verification, rollback triggers, and explicit blast-radiu +0 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build tax report backend aggregating gains/losses and generating filing-year summaries. +1 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +0 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a realtime log viewer backend with tail sessions, regex filters, and backpressure. +1 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Define verification and readiness surface | Build job queue service with delayed jobs, retries, visibility timeout, and metrics. +1 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +1 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +0 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +2 0 Define verification and readiness surface | Build LLM guardrail service for policy checks, PII filtering, and response classification. +0 0 Define verification and readiness surface | Build telemetry collector with trace/span ingestion, sampling, and export adapters. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Define verification and readiness surface | Build a secret rotation controller with staged rollout, canary verification, rollback triggers, and explicit blast-radiu +2 0 Plan Upgrade API contract from v1 to v2 backend | Upgrade API contract from v1 to v2: backend handlers, OpenAPI schema, generated client SDK, frontend integration, and co +2 0 Define verification and readiness surface | Build configuration service with versioned configs, staged rollout, and validation checks. +2 0 Define verification and readiness surface | Build access control service with RBAC roles, permissions, and resource-scoped grants. +2 0 Define verification and readiness surface | Build a feature flag API with environments, targeting rules, and rollout percentages. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +0 0 Define verification and readiness surface | Build an event deduplicator using idempotency keys, time windows, and replay-safe writes. +1 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. diff --git a/specialists/data/generated/automatability_eval.tsv b/specialists/data/generated/automatability_eval.tsv new file mode 100644 index 0000000..356e6e6 --- /dev/null +++ b/specialists/data/generated/automatability_eval.tsv @@ -0,0 +1,46 @@ +2 0 name=LibrarySymbolAdvisor +3 0 name=SelfHostTranspileAudit +1 0 name=TrainingDataGenerator +3 0 name=IntentTranslator +3 0 name=TypeSystemTranslator +0 0 name=ContextBundleMinimizer +3 0 name=SecurityPreservingTranslation +3 0 name=IncrementalOptimizer +0 0 name=BufferOpsLifecycle +0 0 name=MigrationAcceptanceContract +5 0 name=ApprovalEscalationPlanner +0 0 name=Sprint163IntegrationSummary +5 0 name=ArchitectReviewInterface +3 0 name=MemoryModelTranslator +4 0 name=AgentCodeGen +0 0 name=Sprint238IntegrationSummary +1 0 name=CppJSBindingEmitter +2 0 name=OperationClassifier +0 0 name=Sprint36IntegrationSummary +0 0 name=SelfHostHarness +2 0 name=SymbolSelectorAPI +5 0 name=ArchitectReviewSurface +0 0 name=Sprint111IntegrationSummary +2 0 name=LibrarySymbolAdvisor +2 0 name=TaskitemLibraryAnnotator +1 0 name=SyntaxHighlighterElisp +0 0 name=SelfHostCodebaseAudit +2 0 name=PerTaskLibrarySelector +1 0 name=SyntaxHighlighterGo +4 0 name=AgentRegistry +2 0 name=OperationClassifier +1 0 name=WorkflowTemplates +5 0 name=AgentMutationApproval +5 0 name=AgentMutationPreview +1 0 name=RustPythonBindingEmitter +3 0 name=TransformEngine +2 0 name=OperationClassifier +0 0 name=CompilerPhaseSpec +5 0 name=ArchitectReviewSurface +2 0 name=OperationSelectorAPI +5 0 name=AgentMutationPreview +5 0 name=AgentMutationApproval +4 0 name=AgentTaskStatusOverlay +4 0 name=ScopeMilestoneDecomposer +5 0 name=SecurityExceptionReviewBoard +1 0 name=CompositionBuilder diff --git a/specialists/data/generated/automatability_train.tsv b/specialists/data/generated/automatability_train.tsv new file mode 100644 index 0000000..9555984 --- /dev/null +++ b/specialists/data/generated/automatability_train.tsv @@ -0,0 +1,266 @@ +4 0 name=AgentChatContextInjector +5 0 name=ApprovalEscalationPlanner +3 0 name=StrategyAwareOptimizer +0 0 name=SettingsManager +2 0 name=LibrarySymbolAdvisor +3 0 name=TransformEngineExtended +3 0 name=StrategyAwareOptimizer +2 0 name=SymbolSelectorAPI +5 0 name=ApprovalEscalationPlanner +0 0 name=PolyApiProject +3 0 name=MemoryModelTranslator +4 0 name=AgentChatSessionPersistence +4 0 name=AgentSessionRecorder +3 0 name=IncrementalOptimizer +3 0 name=TypeSystemTranslator +1 0 name=MigrationTestGenerator +1 0 name=SCIPEmitter +5 0 name=ArchitectReviewInterface +1 0 name=CapabilityDeclarationGenerator +0 0 name=Sprint190IntegrationSummary +5 0 name=SecurityExceptionReviewBoard +1 0 name=SecurityTestSkeletonGenerator +1 0 name=SchemaToCppGenerator +1 0 name=JobDispatchTableGenerator +0 0 name=Sprint85IntegrationSummary +2 0 name=SymbolSelectorAPI +1 0 name=ProjectAstCMakeGenerator +0 0 name=BatchMutationAPI +2 0 name=CanaryPromotionJudge +5 0 name=ArchitectReviewSurface +3 0 name=TypeAwareMappings +3 0 name=SecurityPreservingTranslation +3 0 name=TransformEngineExtended +5 0 name=ArchitectReviewSurface +0 0 name=Sprint222IntegrationSummary +5 0 name=ArchitectReviewInterface +5 0 name=ArchitectReviewSurface +4 0 name=PolyglotSuiteOrchestrator +0 0 name=Sprint265IntegrationSummary +1 0 name=SyntaxHighlighterOrg +3 0 name=TransformEngineExtended +3 0 name=TypeAwareMappings +4 0 name=AgentChatPanelModel +4 0 name=ArchitectModuleDecomposer +0 0 name=ContextSliceAssembler +1 0 name=SyntaxHighlighterJavaScript +3 0 name=TranspilationConfidence +4 0 name=ArchitectTechStackSelector +2 0 name=PerTaskLibrarySelector +5 0 name=ArchitectReviewInterface +5 0 name=SecurityExceptionReviewBoard +0 0 name=Sprint106IntegrationSummary +2 0 name=OperationClassifier +1 0 name=SQLiteDataLayerGenerator +3 0 name=SelfHostTranspileAudit +0 0 name=Sprint27IntegrationSummary +3 0 name=IncrementalOptimizer +5 0 name=AgentMutationApproval +0 0 name=Sprint98IntegrationSummary +2 0 name=OperationSelectorAPI +3 0 name=SelfHostTranspileAudit +2 0 name=OperationSelectorAPI +0 0 name=AccessibilityBaseline +5 0 name=AgentMutationApproval +1 0 name=TraceGenerator +0 0 name=CostEstimator +4 0 name=AgentPermissionPolicy +5 0 name=AgentMutationPreview +3 0 name=ConcurrencyTranslator +3 0 name=SelfHostTranspileAudit +3 0 name=ContextWindowOptimizer +5 0 name=AgentMutationPreview +3 0 name=IntentTranslator +4 0 name=AgentLibraryPolicy +1 0 name=SyntaxHighlighterPython +1 0 name=JobDispatchTableGenerator +0 0 name=EditorModePolicy +2 0 name=LibrarySymbolAdvisor +4 0 name=AgentMarketplace +3 0 name=TranspilationConfidence +0 0 name=SwarmStatusPanelModel +5 0 name=ApprovalEscalationPlanner +3 0 name=TransformEngine +1 0 name=SyntaxHighlighter +4 0 name=ArchitectIntakeProcessor +4 0 name=PolyglotOrchestrator +4 0 name=AgentTaskStatusOverlay +5 0 name=ArchitectReviewInterface +2 0 name=OperationSelectorAPI +2 0 name=LibrarySymbolAdvisor +0 0 name=Sprint184IntegrationSummary +5 0 name=SecurityExceptionReviewBoard +0 0 name=Sprint36aIntegration +5 0 name=ArchitectReviewInterface +1 0 name=SyntaxHighlighterPython +0 0 name=SearchUtils +0 0 name=Diagnostics +1 0 name=TestPlanGenerator +5 0 name=SecurityExceptionReviewBoard +1 0 name=InferenceJobGenerator +4 0 name=ArchitectProblemParser +0 0 name=LSPSymbolExtractor +2 0 name=SymbolSelectorAPI +4 0 name=ArchitectProjectPipeline +2 0 name=PerTaskLibrarySelector +4 0 name=AgentToolCallVisualization +1 0 name=ProjectSkeletonGenerator +2 0 name=TaskitemLibraryAnnotator +2 0 name=CanaryPromotionJudge +5 0 name=SecurityExceptionReviewBoard +2 0 name=OperationSelectorAPI +1 0 name=LanguageGeneratorPluginManifest +3 0 name=TransformEngine +4 0 name=OnCallCoveragePlanner +2 0 name=TaskitemLibraryAnnotator +2 0 name=TaskitemLibraryAnnotator +5 0 name=AgentMutationPreview +4 0 name=AgentChatSessionPersistence +1 0 name=CHeaderEmitter +0 0 name=Sprint49IntegrationSummary +3 0 name=ContextWindowOptimizer +2 0 name=PerTaskLibrarySelector +4 0 name=ArchitectMultiLanguageOrchestrator +0 0 name=MCPWorkflowPrompts +3 0 name=TranspilationRPC +3 0 name=ContextWindowOptimizer +2 0 name=OperationSelectorAPI +0 0 name=Sprint198IntegrationSummary +4 0 name=ArchitectScaffoldGenerator +1 0 name=GoCppBindingEmitter +0 0 name=AnimationUtils +0 0 name=SelfHostAnnotationAudit +4 0 name=ArchitectIntakeProcessor +4 0 name=AgentMarketplace +3 0 name=TranspilationConfidence +5 0 name=ArchitectReviewSurface +1 0 name=SecureByDefaultGenerator +2 0 name=OperationSelectorAPI +2 0 name=CanaryPromotionJudge +2 0 name=TaskitemLibraryAnnotator +4 0 name=AgentChatPanelModel +2 0 name=LibrarySymbolAdvisor +3 0 name=SecurityPreservingTranslation +0 0 name=APIBoundaryPreserver +5 0 name=ApprovalEscalationPlanner +3 0 name=TypeSystemTranslator +3 0 name=StrategyAwareOptimizer +3 0 name=TransformEngineExtended +1 0 name=SyntaxHighlighterRust +4 0 name=AgentSessionRecorder +0 0 name=Sprint136IntegrationSummary +0 0 name=OperationTaxonomy +4 0 name=ArchitectTechStackSelector +0 0 name=AnnotationConflict +2 0 name=CanaryPromotionJudge +5 0 name=ApprovalEscalationPlanner +0 0 name=Sprint286IntegrationSummary +1 0 name=MigrationPlanGenerator +0 0 name=Sprint261IntegrationSummary +1 0 name=TrainingDataGenerator +1 0 name=RustPythonBindingEmitter +4 0 name=AgentTaskSlots +4 0 name=AgentCodeGen +0 0 name=Sprint74IntegrationSummary +1 0 name=ArchitectTemplates +2 0 name=SymbolSelectorAPI +2 0 name=TaskitemLibraryAnnotator +5 0 name=ApprovalEscalationPlanner +2 0 name=CanaryPromotionJudge +5 0 name=AgentMutationApproval +1 0 name=SyntaxHighlighterJava +5 0 name=AgentMutationPreview +5 0 name=AgentMutationPreview +1 0 name=SQLiteDataLayerGenerator +4 0 name=AgentTaskSlots +4 0 name=AgentRPCHandler +3 0 name=MemoryModelTranslator +3 0 name=ConcurrencyTranslator +5 0 name=ArchitectReviewSurface +4 0 name=AgentToolCallVisualization +5 0 name=SecurityExceptionReviewBoard +1 0 name=RustGoBindingEmitter +0 0 name=Sprint58IntegrationSummary +3 0 name=IncrementalOptimizer +0 0 name=Sprint239IntegrationSummary +1 0 name=SyntaxHighlighterElisp +1 0 name=PlatformCMakeTargetGenerator +5 0 name=ApprovalEscalationPlanner +1 0 name=MqttBoilerplateGenerator +2 0 name=PerTaskLibrarySelector +3 0 name=IntentTranslator +5 0 name=SecurityExceptionReviewBoard +1 0 name=KeySymbolRenderer +2 0 name=OperationSelectorAPI +3 0 name=MemoryModelTranslator +5 0 name=ArchitectReviewInterface +1 0 name=ArchitectSkeletonGenerator +2 0 name=OperationClassifier +5 0 name=ApprovalEscalationPlanner +2 0 name=OperationClassifier +1 0 name=PackageScriptGenerator +2 0 name=TaskitemLibraryAnnotator +3 0 name=TypeAwareMappings +5 0 name=ArchitectReviewInterface +1 0 name=GoCppBindingEmitter +5 0 name=AgentMutationApproval +1 0 name=CHeaderEmitter +1 0 name=SyntaxHighlighterOrg +1 0 name=TaskitemGeneratorV2 +0 0 name=Sprint96IntegrationSummary +0 0 name=ScoreClampUtil +4 0 name=AgentRPCHandler +4 0 name=ArchitectScaffoldGenerator +4 0 name=ScopeMilestoneDecomposer +5 0 name=AgentMutationApproval +5 0 name=AgentMutationPreview +4 0 name=AgentPermissionPolicy +2 0 name=PerTaskLibrarySelector +0 0 name=ConstrainedRoutingRulesetExtension +1 0 name=LanguageGeneratorPluginManifest +5 0 name=ArchitectReviewSurface +0 0 name=WorkItem +2 0 name=TaskitemLibraryAnnotator +2 0 name=CanaryPromotionJudge +5 0 name=AgentMutationApproval +2 0 name=LibrarySymbolAdvisor +3 0 name=ConcurrencyTranslator +2 0 name=CanaryPromotionJudge +4 0 name=WorkflowOrchestrator +1 0 name=InferenceJobGenerator +3 0 name=TranspilationRPC +4 0 name=AgentLibraryPolicy +0 0 name=Sprint272IntegrationSummary +3 0 name=TransformEngine +3 0 name=TranspilationRPC +4 0 name=OnCallCoveragePlanner +2 0 name=SymbolSelectorAPI +4 0 name=PolyglotOrchestrator +2 0 name=OperationClassifier +0 0 name=RefactorActions +2 0 name=LibrarySymbolAdvisor +3 0 name=IntentTranslator +2 0 name=PerTaskLibrarySelector +4 0 name=WorkflowOrchestrator +4 0 name=AgentChatContextInjector +4 0 name=AgentAnnotationAssistant +5 0 name=AgentMutationApproval +2 0 name=SymbolSelectorAPI +5 0 name=ArchitectReviewInterface +1 0 name=SyntaxHighlighterCpp +4 0 name=ArchitectMultiLanguageOrchestrator +1 0 name=APIDocGenerator +4 0 name=ArchitectProblemParser +3 0 name=StrategyAwareOptimizer +4 0 name=AgentAnnotationAssistant +1 0 name=PackageScriptGenerator +4 0 name=AgentRegistry +4 0 name=ArchitectModuleDecomposer +0 0 name=Sprint249IntegrationSummary +3 0 name=SecurityPreservingTranslation +4 0 name=ArchitectProjectPipeline +0 0 name=WorkerDispatchValidator +3 0 name=ContextWindowOptimizer +4 0 name=PolyglotSuiteOrchestrator +5 0 name=AgentMutationPreview +3 0 name=TypeSystemTranslator diff --git a/specialists/data/generated/confidence_tier_eval.tsv b/specialists/data/generated/confidence_tier_eval.tsv new file mode 100644 index 0000000..9ef70ab --- /dev/null +++ b/specialists/data/generated/confidence_tier_eval.tsv @@ -0,0 +1,180 @@ +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=0 prereqs=4 queueready=no +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=2 prereqs=1 queueready=no +2 0 conflicts=6 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=4 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=2 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=1 deps=0 prereqs=3 queueready=no +2 0 conflicts=4 ambiguity=0 deps=1 prereqs=3 queueready=no +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=2 deps=2 prereqs=3 queueready=no +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=0 prereqs=1 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=5 ambiguity=2 deps=2 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=0 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=0 deps=0 prereqs=3 queueready=no +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=0 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=3 deps=0 prereqs=3 queueready=no +2 0 conflicts=2 ambiguity=1 deps=2 prereqs=3 queueready=no +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=3 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=4 queueready=yes +2 0 conflicts=7 ambiguity=3 deps=1 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=0 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=2 prereqs=4 queueready=no +2 0 conflicts=4 ambiguity=2 deps=2 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=2 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=0 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=2 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=4 queueready=yes +2 0 conflicts=1 ambiguity=1 deps=2 prereqs=3 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=no +2 0 conflicts=1 ambiguity=2 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=1 deps=0 prereqs=3 queueready=no +2 0 conflicts=0 ambiguity=2 deps=1 prereqs=2 queueready=no +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=1 prereqs=2 queueready=no +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=2 deps=1 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=1 prereqs=2 queueready=no +2 0 conflicts=9 ambiguity=3 deps=2 prereqs=3 queueready=no +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=0 deps=1 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=3 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=1 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=1 deps=1 prereqs=4 queueready=no +2 0 conflicts=8 ambiguity=2 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=3 deps=0 prereqs=4 queueready=no +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=3 ambiguity=2 deps=1 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=no +2 0 conflicts=8 ambiguity=2 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=3 deps=2 prereqs=2 queueready=yes diff --git a/specialists/data/generated/confidence_tier_train.tsv b/specialists/data/generated/confidence_tier_train.tsv new file mode 100644 index 0000000..05dea7e --- /dev/null +++ b/specialists/data/generated/confidence_tier_train.tsv @@ -0,0 +1,1020 @@ +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=3 queueready=yes +2 0 conflicts=1 ambiguity=0 deps=1 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=0 prereqs=2 queueready=no +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=2 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=1 prereqs=1 queueready=no +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=1 deps=1 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=2 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=3 queueready=yes +2 0 conflicts=6 ambiguity=1 deps=0 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=1 deps=0 prereqs=4 queueready=no +2 0 conflicts=4 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=2 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=1 deps=1 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=4 queueready=yes +2 0 conflicts=9 ambiguity=3 deps=1 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=1 ambiguity=1 deps=2 prereqs=1 queueready=no +2 0 conflicts=0 ambiguity=1 deps=1 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=3 queueready=yes +2 0 conflicts=5 ambiguity=1 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=0 prereqs=2 queueready=no +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=6 ambiguity=2 deps=2 prereqs=1 queueready=no +2 0 conflicts=3 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=0 deps=2 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=2 deps=1 prereqs=4 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=2 prereqs=4 queueready=no +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=7 ambiguity=3 deps=2 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=0 deps=1 prereqs=2 queueready=no +2 0 conflicts=6 ambiguity=1 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=0 prereqs=4 queueready=no +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=2 prereqs=4 queueready=no +2 0 conflicts=6 ambiguity=0 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=1 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=2 ambiguity=0 deps=1 prereqs=3 queueready=no +2 0 conflicts=5 ambiguity=3 deps=1 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=1 prereqs=2 queueready=no +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=2 prereqs=4 queueready=no +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=1 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=2 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=1 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=7 ambiguity=3 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=0 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=no +2 0 conflicts=6 ambiguity=3 deps=0 prereqs=4 queueready=no +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=2 queueready=yes +2 0 conflicts=6 ambiguity=2 deps=2 prereqs=3 queueready=no +2 0 conflicts=5 ambiguity=3 deps=2 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=2 deps=1 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=2 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=2 prereqs=3 queueready=no +2 0 conflicts=1 ambiguity=2 deps=0 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=4 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=1 prereqs=1 queueready=no +2 0 conflicts=8 ambiguity=3 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=0 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=5 ambiguity=2 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=1 prereqs=3 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=7 ambiguity=2 deps=2 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=1 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=4 queueready=yes +2 0 conflicts=6 ambiguity=2 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=2 ambiguity=0 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=0 prereqs=3 queueready=no +2 0 conflicts=9 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=2 deps=0 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=no +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=2 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=no +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=2 deps=2 prereqs=4 queueready=no +2 0 conflicts=5 ambiguity=3 deps=0 prereqs=1 queueready=no +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=1 prereqs=4 queueready=no +2 0 conflicts=9 ambiguity=0 deps=2 prereqs=2 queueready=no +2 0 conflicts=1 ambiguity=3 deps=1 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=3 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=2 prereqs=3 queueready=no +2 0 conflicts=0 ambiguity=3 deps=0 prereqs=1 queueready=no +2 0 conflicts=6 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=0 deps=2 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=1 ambiguity=2 deps=2 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=3 queueready=yes +2 0 conflicts=1 ambiguity=0 deps=2 prereqs=2 queueready=no +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=0 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=0 deps=1 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=1 ambiguity=2 deps=1 prereqs=2 queueready=no +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=1 prereqs=2 queueready=no +2 0 conflicts=2 ambiguity=2 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=2 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=1 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=no +2 0 conflicts=3 ambiguity=0 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=no +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=1 prereqs=3 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=0 prereqs=2 queueready=no +2 0 conflicts=4 ambiguity=1 deps=1 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=0 deps=0 prereqs=2 queueready=no +2 0 conflicts=7 ambiguity=0 deps=0 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=3 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=1 prereqs=1 queueready=no +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=0 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=3 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=1 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=1 deps=2 prereqs=4 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=1 prereqs=2 queueready=no +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=3 deps=0 prereqs=4 queueready=no +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=2 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=0 prereqs=2 queueready=yes +2 0 conflicts=7 ambiguity=2 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=2 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=1 prereqs=1 queueready=no +2 0 conflicts=9 ambiguity=3 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=1 prereqs=4 queueready=no +2 0 conflicts=2 ambiguity=3 deps=1 prereqs=3 queueready=no +2 0 conflicts=9 ambiguity=1 deps=1 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=1 prereqs=2 queueready=no +2 0 conflicts=5 ambiguity=3 deps=2 prereqs=4 queueready=no +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=1 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=2 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=0 prereqs=1 queueready=no +2 0 conflicts=5 ambiguity=3 deps=0 prereqs=2 queueready=no +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=2 prereqs=4 queueready=no +2 0 conflicts=2 ambiguity=2 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=1 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=1 deps=2 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=3 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=0 prereqs=3 queueready=no +2 0 conflicts=3 ambiguity=0 deps=1 prereqs=2 queueready=no +2 0 conflicts=6 ambiguity=3 deps=2 prereqs=3 queueready=no +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=7 ambiguity=3 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=6 ambiguity=0 deps=2 prereqs=4 queueready=no +2 0 conflicts=5 ambiguity=1 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=0 deps=1 prereqs=3 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=2 prereqs=4 queueready=no +2 0 conflicts=3 ambiguity=1 deps=2 prereqs=3 queueready=no +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=3 ambiguity=2 deps=2 prereqs=3 queueready=no +2 0 conflicts=8 ambiguity=0 deps=0 prereqs=3 queueready=no +2 0 conflicts=6 ambiguity=2 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=1 prereqs=3 queueready=no +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=0 deps=0 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=2 prereqs=1 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=2 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=2 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=3 queueready=yes +2 0 conflicts=3 ambiguity=1 deps=1 prereqs=4 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=1 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=0 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=4 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=2 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=1 prereqs=2 queueready=no +2 0 conflicts=6 ambiguity=0 deps=0 prereqs=4 queueready=no +2 0 conflicts=9 ambiguity=1 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=4 queueready=yes +2 0 conflicts=7 ambiguity=3 deps=0 prereqs=3 queueready=no +2 0 conflicts=6 ambiguity=1 deps=2 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=2 prereqs=4 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=2 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=2 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=0 prereqs=3 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=2 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=1 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=0 deps=1 prereqs=3 queueready=no +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=2 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=0 prereqs=2 queueready=no +2 0 conflicts=3 ambiguity=2 deps=1 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=1 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=3 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=2 prereqs=4 queueready=no +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=0 deps=2 prereqs=4 queueready=no +2 0 conflicts=1 ambiguity=1 deps=1 prereqs=2 queueready=no +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=3 queueready=yes +2 0 conflicts=3 ambiguity=1 deps=2 prereqs=3 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=0 prereqs=3 queueready=no +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=1 ambiguity=3 deps=0 prereqs=4 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=0 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=1 prereqs=1 queueready=no +2 0 conflicts=4 ambiguity=0 deps=1 prereqs=1 queueready=no +2 0 conflicts=4 ambiguity=0 deps=0 prereqs=1 queueready=no +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=2 prereqs=4 queueready=no +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=2 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=0 prereqs=3 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=3 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=2 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=3 ambiguity=1 deps=2 prereqs=2 queueready=no +2 0 conflicts=6 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=6 ambiguity=2 deps=0 prereqs=3 queueready=no +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=6 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=2 prereqs=4 queueready=no +2 0 conflicts=6 ambiguity=1 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=2 prereqs=3 queueready=no +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=1 queueready=yes +2 0 conflicts=6 ambiguity=2 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=1 prereqs=1 queueready=no +2 0 conflicts=4 ambiguity=0 deps=2 prereqs=1 queueready=no +2 0 conflicts=7 ambiguity=2 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=7 ambiguity=2 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=1 deps=0 prereqs=4 queueready=no +2 0 conflicts=3 ambiguity=3 deps=2 prereqs=3 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=1 prereqs=3 queueready=no +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=2 queueready=yes +2 0 conflicts=7 ambiguity=1 deps=1 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=1 deps=0 prereqs=1 queueready=yes +2 0 conflicts=1 ambiguity=3 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=no +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=4 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=1 prereqs=3 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=3 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=1 prereqs=4 queueready=no +2 0 conflicts=9 ambiguity=3 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=3 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=2 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=1 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=2 prereqs=1 queueready=yes +2 0 conflicts=0 ambiguity=1 deps=2 prereqs=3 queueready=no +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=0 prereqs=2 queueready=no +2 0 conflicts=5 ambiguity=2 deps=2 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=2 deps=0 prereqs=4 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=1 prereqs=4 queueready=no +2 0 conflicts=6 ambiguity=2 deps=1 prereqs=1 queueready=no +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=no +2 0 conflicts=6 ambiguity=2 deps=1 prereqs=3 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=no +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=2 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=2 prereqs=2 queueready=no +2 0 conflicts=4 ambiguity=1 deps=2 prereqs=3 queueready=no +2 0 conflicts=1 ambiguity=1 deps=2 prereqs=2 queueready=no +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=3 queueready=yes +2 0 conflicts=5 ambiguity=1 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=1 deps=0 prereqs=1 queueready=no +2 0 conflicts=6 ambiguity=0 deps=0 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=4 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=2 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=0 prereqs=3 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=1 prereqs=1 queueready=no +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=6 ambiguity=2 deps=2 prereqs=1 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=1 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=3 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=0 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=0 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=0 ambiguity=1 deps=2 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=2 prereqs=4 queueready=no +2 0 conflicts=4 ambiguity=0 deps=1 prereqs=4 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=3 ambiguity=2 deps=1 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=5 ambiguity=2 deps=0 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=4 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=0 prereqs=3 queueready=no +2 0 conflicts=3 ambiguity=3 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=1 deps=0 prereqs=2 queueready=no +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=1 prereqs=1 queueready=no +2 0 conflicts=5 ambiguity=1 deps=2 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=0 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=0 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=2 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=3 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=1 prereqs=3 queueready=no +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=3 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=2 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=1 ambiguity=3 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=3 deps=2 prereqs=2 queueready=no +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=0 prereqs=1 queueready=no +2 0 conflicts=8 ambiguity=3 deps=2 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=1 prereqs=1 queueready=no +2 0 conflicts=3 ambiguity=2 deps=1 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=1 deps=0 prereqs=3 queueready=no +2 0 conflicts=7 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=2 deps=2 prereqs=3 queueready=no +2 0 conflicts=6 ambiguity=0 deps=1 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=2 prereqs=1 queueready=no +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=1 prereqs=1 queueready=no +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=4 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=1 deps=2 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=3 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=2 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=3 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=2 prereqs=2 queueready=no +2 0 conflicts=5 ambiguity=2 deps=0 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=6 ambiguity=0 deps=0 prereqs=3 queueready=no +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=0 prereqs=1 queueready=no +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=2 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=1 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=0 prereqs=2 queueready=no +2 0 conflicts=8 ambiguity=2 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=2 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=2 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=1 deps=0 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=2 ambiguity=1 deps=1 prereqs=3 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=0 deps=1 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=1 deps=2 prereqs=2 queueready=no +2 0 conflicts=2 ambiguity=1 deps=1 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=1 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=3 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=2 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=4 queueready=yes +2 0 conflicts=2 ambiguity=1 deps=2 prereqs=4 queueready=no +2 0 conflicts=8 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=5 ambiguity=3 deps=0 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=3 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=0 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=1 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=1 prereqs=4 queueready=no +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=3 deps=2 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=0 deps=0 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=0 ambiguity=3 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=1 ambiguity=3 deps=1 prereqs=4 queueready=no +2 0 conflicts=3 ambiguity=1 deps=2 prereqs=2 queueready=yes +2 0 conflicts=9 ambiguity=2 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=2 deps=2 prereqs=4 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=2 queueready=yes +2 0 conflicts=1 ambiguity=3 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=5 ambiguity=1 deps=1 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=7 ambiguity=2 deps=1 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +2 0 conflicts=2 ambiguity=0 deps=2 prereqs=2 queueready=no +2 0 conflicts=5 ambiguity=2 deps=2 prereqs=4 queueready=no +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=0 ambiguity=0 deps=1 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=3 deps=1 prereqs=3 queueready=no +2 0 conflicts=0 ambiguity=0 deps=1 prereqs=4 queueready=no +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=2 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=2 queueready=yes +2 0 conflicts=6 ambiguity=2 deps=0 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=0 prereqs=4 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=1 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=1 deps=0 prereqs=4 queueready=no +2 0 conflicts=5 ambiguity=2 deps=1 prereqs=2 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=2 queueready=yes +1 0 conflicts=3 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=7 ambiguity=2 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=2 prereqs=3 queueready=no +2 0 conflicts=4 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=1 prereqs=4 queueready=no +2 0 conflicts=9 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=6 ambiguity=0 deps=2 prereqs=1 queueready=no +1 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=no +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=1 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=9 ambiguity=1 deps=0 prereqs=4 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=3 deps=0 prereqs=4 queueready=no +2 0 conflicts=7 ambiguity=0 deps=1 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=0 ambiguity=2 deps=1 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=0 prereqs=2 queueready=no +2 0 conflicts=3 ambiguity=0 deps=2 prereqs=3 queueready=no +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=2 queueready=yes +2 0 conflicts=2 ambiguity=3 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=9 ambiguity=3 deps=0 prereqs=1 queueready=no +2 0 conflicts=5 ambiguity=1 deps=2 prereqs=3 queueready=yes +2 0 conflicts=9 ambiguity=0 deps=2 prereqs=1 queueready=yes +2 0 conflicts=8 ambiguity=0 deps=1 prereqs=3 queueready=yes +1 0 conflicts=1 ambiguity=0 deps=0 prereqs=3 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=1 prereqs=3 queueready=no +1 0 conflicts=0 ambiguity=1 deps=1 prereqs=3 queueready=yes +2 0 conflicts=1 ambiguity=2 deps=0 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=1 queueready=yes +2 0 conflicts=6 ambiguity=3 deps=1 prereqs=1 queueready=no +1 0 conflicts=1 ambiguity=0 deps=1 prereqs=4 queueready=yes +2 0 conflicts=1 ambiguity=3 deps=0 prereqs=1 queueready=no +1 0 conflicts=2 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=1 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=2 prereqs=2 queueready=yes +2 0 conflicts=6 ambiguity=1 deps=0 prereqs=4 queueready=no +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=2 queueready=yes +2 0 conflicts=5 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=1 queueready=yes +1 0 conflicts=1 ambiguity=1 deps=2 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=1 queueready=yes +2 0 conflicts=3 ambiguity=2 deps=1 prereqs=3 queueready=no +1 0 conflicts=2 ambiguity=0 deps=0 prereqs=2 queueready=yes +1 0 conflicts=0 ambiguity=1 deps=0 prereqs=4 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=0 prereqs=1 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=2 prereqs=1 queueready=yes +1 0 conflicts=0 ambiguity=2 deps=1 prereqs=2 queueready=yes +2 0 conflicts=4 ambiguity=1 deps=2 prereqs=3 queueready=yes +2 0 conflicts=2 ambiguity=2 deps=1 prereqs=2 queueready=yes +2 0 conflicts=8 ambiguity=3 deps=2 prereqs=3 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=1 prereqs=2 queueready=yes +0 0 conflicts=0 ambiguity=0 deps=0 prereqs=4 queueready=yes diff --git a/specialists/data/generated/prereq_op_eval.tsv b/specialists/data/generated/prereq_op_eval.tsv new file mode 100644 index 0000000..b4448c6 --- /dev/null +++ b/specialists/data/generated/prereq_op_eval.tsv @@ -0,0 +1,24 @@ +2 0 `whetstone_enqueue_pair_upgrade` MCP tool +3 0 Deprecation/succession lifecycle framework +3 0 Security scan orchestrator for generated targets +1 0 Phase 12c Integration Tests +2 0 Reviewer quorum policy model for critical approvals +1 0 Phase 38a Integration +1 0 Phase 23b Integration + Sprint Summary +0 0 Reprojection safety policy +2 0 Stable corpus certification suite +0 0 Timing regression runner integration +0 0 `whetstone_get_tool_permissions` MCP tool +3 0 Cross-language rollout state snapshot model +1 0 Compatibility policy profiles by interface criticality +3 0 `whetstone_publish_roadmap_epoch` MCP tool +2 0 Security Report Generation +2 0 Compatibility/deprecation report artifact +1 0 Human Review Interface via MCP +2 0 HiveMind job publisher from editor +1 0 Parser/AST adapter interface schema +0 0 Sprint 76 integration summary + regression +1 0 Scenario — Multi-Model Orchestration +0 0 Autonomous loop controller with “green or explicit blocked” contract +2 0 Security-Aware Routing +0 0 Verifier trust weighting policy engine diff --git a/specialists/data/generated/prereq_op_projects_eval.tsv b/specialists/data/generated/prereq_op_projects_eval.tsv new file mode 100644 index 0000000..e3952e8 --- /dev/null +++ b/specialists/data/generated/prereq_op_projects_eval.tsv @@ -0,0 +1,12 @@ +0 0 Define verification and readiness surface | Build distributed rate limiter with fixed/sliding window modes and policy APIs. +1 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +1 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +2 0 Plan Replace search ranking pipeline behind feature flag with | Replace search ranking pipeline behind feature flag with staged rollout, telemetry checks, and automatic abort trigger. +0 0 Define verification and readiness surface | Build a JWT auth service with register, login, refresh token rotation, and password hashing. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua diff --git a/specialists/data/generated/prereq_op_projects_train.tsv b/specialists/data/generated/prereq_op_projects_train.tsv new file mode 100644 index 0000000..eb717b2 --- /dev/null +++ b/specialists/data/generated/prereq_op_projects_train.tsv @@ -0,0 +1,68 @@ +0 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Define verification and readiness surface | Build chat bot backend with slash commands, permission checks, and scheduled digests. +1 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +2 0 Define verification and readiness surface | Build a CMS backend with posts, tags, draft/publish workflow, and role-based access control. +0 0 Define verification and readiness surface | Build a multi-tenant limiter with token bucket + sliding window hybrid policy, burst fairness guarantees, and lock-free +1 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +2 0 Define verification and readiness surface | Build a CLI password vault with encrypted local storage and master-key unlock. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Define verification and readiness surface | Build a release notes generator from commits, PR labels, and issue references. +2 0 Define verification and readiness surface | Build a secrets rotation scheduler with lease tracking, retry plans, and notifications. +1 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +2 0 Define verification and readiness surface | Build a model registry API supporting stages, approvals, rollback, and signed artifact links. +1 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +1 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +1 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +1 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +0 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Define verification and readiness surface | Build inventory endpoints with stock reservation, release, and low-stock alerts. +0 0 Define verification and readiness surface | Build form builder backend with schema validation, submissions, and analytics counters. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +1 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +0 0 Define verification and readiness surface | Build experiment tracking backend storing params, metrics, artifacts, and compare queries. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +0 0 Define verification and readiness surface | Build a .env manager CLI with profile switch, validation, and secure redaction in output. +0 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Define verification and readiness surface | Build a secret rotation controller with staged rollout, canary verification, rollback triggers, and explicit blast-radiu +0 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build tax report backend aggregating gains/losses and generating filing-year summaries. +1 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +0 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a realtime log viewer backend with tail sessions, regex filters, and backpressure. +1 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Define verification and readiness surface | Build job queue service with delayed jobs, retries, visibility timeout, and metrics. +1 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +1 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +0 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +2 0 Define verification and readiness surface | Build LLM guardrail service for policy checks, PII filtering, and response classification. +0 0 Define verification and readiness surface | Build telemetry collector with trace/span ingestion, sampling, and export adapters. +3 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Define verification and readiness surface | Build a secret rotation controller with staged rollout, canary verification, rollback triggers, and explicit blast-radiu +2 0 Plan Upgrade API contract from v1 to v2 backend | Upgrade API contract from v1 to v2: backend handlers, OpenAPI schema, generated client SDK, frontend integration, and co +2 0 Define verification and readiness surface | Build configuration service with versioned configs, staged rollout, and validation checks. +2 0 Define verification and readiness surface | Build access control service with RBAC roles, permissions, and resource-scoped grants. +2 0 Define verification and readiness surface | Build a feature flag API with environments, targeting rules, and rollout percentages. +3 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +0 0 Define verification and readiness surface | Build an event deduplicator using idempotency keys, time windows, and replay-safe writes. +1 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. diff --git a/specialists/data/generated/prereq_op_train.tsv b/specialists/data/generated/prereq_op_train.tsv new file mode 100644 index 0000000..56e0813 --- /dev/null +++ b/specialists/data/generated/prereq_op_train.tsv @@ -0,0 +1,136 @@ +2 0 `whetstone_get_constructive_rollout_status` runtime implementation +1 0 Target projection blocker taxonomy +2 0 Strict gate mode in production loop +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +3 0 Security scan orchestrator for generated targets +1 0 Multi-language constructive loop report artifact +2 0 `whetstone_get_constructive_rollout_status` MCP tool +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +2 0 Deprecation policy model for unstable/obsolete pairs +0 0 Projection schema extension for run catalogs +0 0 Panel Registration System +0 0 Sprint 146 integration summary + regression +3 0 `whetstone_publish_roadmap_epoch` MCP tool +1 0 Distributed path in FeedbackLoopOrchestrator +1 0 Continuity drill framework and scoring +3 0 Cross-language release evidence aggregator model +0 0 Scheme lowering/raising adapters +3 0 Cross-language release evidence aggregator model +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +1 0 Phase 13a Integration — Layout Persistence +2 0 Security Testing Skeleton Generation +0 0 Modifier-Edge Shortcut Notation +1 0 Architecture Templates +3 0 Deprecation/succession lifecycle framework +1 0 Language-specific method body scaffolding for core patterns +1 0 Initializer Lists + STL Patterns +3 0 Security scan orchestrator for generated targets +0 0 `whetstone_dry_run_patch` MCP tool +3 0 Cross-language rollout state snapshot model +3 0 Security scan orchestrator for generated targets +0 0 Cost telemetry integration in metrics stack +2 0 Constitutional invariants compliance checker +0 0 Property-based equivalence runner +2 0 Regional rollout profile integration +0 0 Compile/test proof binding +3 0 Cross-language release evidence aggregator model +2 0 `whetstone_get_certification_status` MCP tool +3 0 Deprecation/succession lifecycle framework +0 0 Sprint 193 Integration Summary +1 0 Multi-language replay corpus runtime index model +2 0 Benchmark suite for production capability claim +3 0 `whetstone_publish_roadmap_epoch` MCP tool +0 0 Build output parser — GCC/Clang format +2 0 Rollout stage model (`pilot`, `canary`, `broad`, `default`) +2 0 `whetstone_set_language_rollout_tier` MCP tool +0 0 Breadcrumb Navigation +1 0 Runtime-aware lowering extension points +1 0 Phase 15b Integration — Full Protocol Test +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +0 0 Python lowering adapter v2 (dynamic shape packets) +0 0 Canonical fixture and oracle dataset packs +0 0 Project context auto-injection +0 0 LTS baseline criteria schema +3 0 Security scan orchestrator for generated targets +0 0 `whetstone_validate_tool_contracts` MCP tool +1 0 Phase 20a Integration +3 0 Cross-language release evidence aggregator model +2 0 `whetstone_publish_next_block_plan` MCP tool +2 0 Staged cutover planner (`shadow`, `dual-run`, `cutover`) +3 0 Security scan orchestrator for generated targets +3 0 Security scan orchestrator for generated targets +0 0 Locals + Watches Panel Data Model +0 0 Context Bundle Minimizer +2 0 `whetstone_get_compliance_evidence` MCP tool +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +2 0 Security Annotation Types — Subject 10 +0 0 Benchmark scorecard generator per pair +1 0 Runtime-specific verification plugin interface +2 0 Certification job scheduler +3 0 Deprecation/succession lifecycle framework +2 0 Tiered rollout and rollback control model +2 0 Sprint 60 integration summary + release marker +3 0 Cross-language rollout state snapshot model +0 0 Constraint Violation Diagnostics +1 0 Phase 12a Integration Tests +3 0 Cross-language rollout state snapshot model +1 0 Deterministic blocker taxonomy extension +0 0 `whetstone_get_regeneration_decisions` MCP tool +1 0 Long-horizon roadmap planning engine v2 +0 0 Intake extraction confidence scoring +1 0 Phase 18b Integration + Sprint Summary +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +2 0 Certification badges for roles/workflows +0 0 Closeout consistency gate +3 0 Cross-language rollout state snapshot model +1 0 Binary/interface diff engine integration +2 0 Release Readiness Gate Pack +0 0 Source-vs-runtime drift comparator model +1 0 Phase 12d Integration + Sprint 12 Summary +1 0 `whetstone_verify_architecture_consistency` MCP tool +0 0 API/ABI preservation report artifact +0 0 Sprint 164 Integration Summary +3 0 `whetstone_publish_roadmap_epoch` MCP tool +0 0 `whetstone_capture_mcp_execution_trace` MCP tool +0 0 Template recommendation model by project fingerprint +0 0 Kotlin Generator +3 0 Deprecation/succession lifecycle framework +1 0 Cross-language capability contract normalizer model +2 0 Release certification packet model +2 0 Rollout blast-radius estimator +3 0 Cross-language release evidence aggregator model +2 0 Tenant-scoped waiver/approval ledger +1 0 Governance model refresh framework +2 0 `whetstone_publish_next_epoch_plan` MCP tool +3 0 Cross-language release evidence aggregator model +0 0 Embedded migration risk report template +1 0 Cross-language node identity preservation policy model +0 0 Key Symbol Rendering +1 0 Burn-down planning optimizer +3 0 Deprecation/succession lifecycle framework +2 0 Interop certification policy bindings +3 0 Cross-language release evidence aggregator model +3 0 Deprecation/succession lifecycle framework +2 0 Production summary hardening +3 0 `whetstone_publish_roadmap_epoch` MCP tool +2 0 Safety-critical deployment dossier artifact +3 0 `whetstone_publish_roadmap_epoch` MCP tool +2 0 Security-Preserving Translation +1 0 Swarm maintenance job decomposition model +3 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +1 0 Polyglot project migration graph model +1 0 Phase 10d Integration Tests +3 0 Cross-language rollout state snapshot model +1 0 Phase 18a Integration +2 0 Runtime-backed constructive release certification packet model +1 0 Phase 29a Integration +2 0 Production loop contract preflight +1 0 Phase 20b Integration + Sprint Summary +2 0 Pair-upgrade queue model +2 0 `whetstone_get_tool_deprecations` MCP tool +0 0 Wire `ArchitectIntakeProcessor` into `whetstone_architect_intake` handler +2 0 Contract drift classifier (`compatible`, `warning`, `breaking`) +1 0 Phase 23c Integration (tests: integration, regression) +1 0 `FeedbackLoopOrchestrator` core loop +1 0 WAT Cross-Language Projection +0 0 Sprint 187 Integration Summary diff --git a/specialists/data/generated/verification_type_eval.tsv b/specialists/data/generated/verification_type_eval.tsv new file mode 100644 index 0000000..f7b3bc4 --- /dev/null +++ b/specialists/data/generated/verification_type_eval.tsv @@ -0,0 +1,172 @@ +2 0 `whetstone_apply_text_ast_merge` MCP tool +2 0 Role-based onboarding model +4 0 Security hardening report artifact +3 0 Governance and audit report bundle +0 0 VB.NET lowering/raising adapters +4 0 Policy tuning report artifact +1 0 `whetstone_get_learning_progress` MCP tool +4 0 Family-level acceptance report generator +3 0 Sprint 178 Integration Summary +1 0 Leak/Corruption Signal Bridge +3 0 Regression fixtures for under-constrained vs capability-limited cases +2 0 `TestFailureRecord` schema +2 0 MCP tool contract schema v1 model +1 0 `whetstone_get_debug_evidence_index` MCP tool +2 0 Error type synthesis for drone +3 0 Sprint 154 integration summary + regression +4 0 `whetstone_publish_next_block_plan` MCP tool +1 0 `RegisterCodegenTools.h` — whetstone_generate_dispatch_table +2 0 `whetstone_get_constructive_status` MCP tool +0 0 Generator class-child traversal fix pack +0 0 Theme System +4 0 API/ABI preservation report artifact +3 0 Sprint 138 integration summary + regression +2 0 Safety case schema (`claim`, `evidence`, `assumption`, `risk`) +3 0 Sprint 112 integration summary + regression +3 0 Sprint 96 integration summary + regression +1 0 `whetstone_rank_failure_triage_actions` MCP tool +1 0 Disassembly + Register View Baseline +3 0 Sprint 94 integration summary + regression +4 0 Runtime selection report artifact +2 0 Diagram/spec artifact ingestion adapters +4 0 Program benchmarking report bundle +2 0 Backward-compatibility serialization path +1 0 Forbidden API policy checker +4 0 Hybrid mode readiness report artifact +4 0 Swarm maintenance report artifact +0 0 `resource_conflict_unresolved` warning class +2 0 Resilience and continuity requirements model +1 0 `whetstone_get_region_policy_profile` MCP tool +3 0 Sprint 117 integration summary + regression +3 0 Sprint 79 integration summary + regression +0 0 Staged cutover planner (`shadow`, `dual-run`, `cutover`) +4 0 Equivalence evidence bundle exporter +2 0 Numerical stability canonical model +4 0 Drift outlook report artifact +1 0 Phase 31a Integration +3 0 API behavior regression runner integration +2 0 Residual risk ownership model +4 0 Documentation pack for language support + migration playbooks +0 0 Marketplace policy bindings for tenants +1 0 Compliance gate integration with support tiers +2 0 Result-to-ObservationRecord mapper +2 0 Multimodal evidence completeness scoring model +0 0 Pattern library governance workflow +0 0 Breadcrumb Navigation +2 0 Conflict region detector model +3 0 Sprint 148 integration summary + regression +4 0 Public spec for verification evidence bundles +3 0 Sprint 116 integration summary + regression +1 0 Phase 11a Integration Tests +0 0 Context slice canonicalization for handoffs +0 0 Approval chain policy engine +4 0 MCP closure publication bundle +3 0 Sprint 132 integration summary + regression +3 0 Sprint 207 Integration Summary +3 0 Sprint 62 integration summary + regression +0 0 Cost Estimation + Optimization +0 0 Theme Token System v2 +2 0 GA evidence aggregation runtime model +1 0 `whetstone_set_workstream_capacity` MCP tool +2 0 Multi-check gate orchestrator model +0 0 Compile/test proof binding +0 0 Semantic pass aggregation in summaries +1 0 `whetstone_transpile_dynamic_family` MCP tool +3 0 Century retrospective evidence bundle composer +2 0 `whetstone_rollback_last_patch` MCP tool +0 0 Policy inheritance/override resolution engine +2 0 `whetstone_get_constructive_rollout_status` MCP tool +2 0 Polyglot project migration graph model +1 0 Phase 12d Integration + Sprint 12 Summary +3 0 Dependency and supply-chain audit packet +0 0 Swarm-to-governance gate adapter +1 0 Allocation/Ownership Trace Hooks +2 0 Failover orchestration policy model +4 0 Self-Hosting Metrics Report +4 0 Multi-adapter compatibility report artifact +4 0 Handoff integrity report artifact +4 0 Campaign report bundle generator +0 0 Supply-chain integrity checks for marketplace artifacts +1 0 Real regenerate tool binding +3 0 Sprint 66 integration summary + regression +4 0 Lock metadata documentation packet +3 0 Sprint 186 Integration Summary +0 0 Kanban-Style Task Board +4 0 Route integrity report artifact +1 0 `whetstone_plan_rollout_stage` MCP tool +4 0 Compatibility/deprecation report artifact +3 0 Sprint 139 integration summary + regression +0 0 Parse AnnotationConflictExtended.h +3 0 Sprint 88 integration summary + regression +2 0 Campaign checkpoint model +4 0 Guided Architect-to-Execution Demo Mode +2 0 `whetstone_resume_constructive_transaction` MCP tool +0 0 SQL Annotation Mapping +1 0 `whetstone_get_program_kpis` MCP tool +3 0 Regression guard planner +1 0 Tier promotion gate runner integration +1 0 `whetstone_label_patch_risk` MCP tool +1 0 `whetstone_parse_build_output` MCP tool +4 0 Runtime freshness report artifact +2 0 Campaign progress tracker model +2 0 MCP closure gate aggregate schema +4 0 Multi-adapter compatibility report artifact +1 0 Snapshot persistence (sidecar integration) +2 0 Prolog lowering adapter v1 (query/backtracking model) +2 0 Efficiency metric model +4 0 LTS publication bundle + operator handbook +4 0 `whetstone_publish_next_block_plan` MCP tool +1 0 `whetstone_verify_financial_migration` MCP tool +0 0 Lint diagnostics normalization +0 0 Dependency Graph Data +4 0 MCP closure publication bundle +1 0 Tamper detection and forensic event pipeline +1 0 Sprint 40 Integration + Final Summary +0 0 Async/await state intent lowering +1 0 `whetstone_get_debt_inventory` MCP tool +0 0 Runtime-aware raising policy selectors +3 0 Systemic regression detector +3 0 Phase 20b Integration + Sprint Summary +0 0 Review-Gate Policy Refinement +4 0 Low-level acceptance report + waiver system +4 0 Conflict/merge report artifact +0 0 Borrow semantics to references/views strategy +2 0 `EnvironmentDiff` schema and computation +0 0 Failure-class-driven patch proposer +0 0 Internal implementation conformance runner +2 0 Generic regeneration policy schema +3 0 Sprint 201 Integration Summary +3 0 Sprint 150 integration summary + full-program regression +4 0 Enterprise rollout report template +4 0 Federated evidence bundle composer +4 0 Cross-language constructive execution report artifact +1 0 Job submission via nexus HTTP API +2 0 `resourceLocks` schema extension +0 0 Assurance review dossier template +4 0 Reliability and rollback report artifact +3 0 Sprint 87 integration summary + regression +2 0 `DistributedFailureEvidence` schema + normalizer +3 0 Sprint 172 Integration Summary +0 0 Exception + Stack Trace Capture +4 0 Conflict/merge qualification report artifact +1 0 CI integration in build/test flow +3 0 Phase 23c Integration (tests: integration, regression) +0 0 Next-epoch strategy generator +0 0 `whetstone_normalize_diagnostics` runtime implementation +4 0 Interop publication bundle +0 0 Auto-generated mitigation suggestions for breaches +1 0 `whetstone_propose_patch_for_failure` MCP tool +3 0 Phase 22b Integration + Sprint Summary +3 0 Strictness audit utility and baseline report +2 0 Planner-executor handoff packet +1 0 `whetstone_cluster_failures` MCP tool +4 0 Pattern efficacy report artifact +1 0 `whetstone_run_certification_cycle` MCP tool +1 0 `whetstone_verify_executable_equivalence` MCP tool +4 0 Guided transpilation exercise framework +1 0 `whetstone_get_top_adapter_gaps` MCP tool +2 0 Skeleton AST Support +1 0 `whetstone_get_lts_support_matrix` MCP tool +4 0 Documentation pack for language support + migration playbooks +4 0 Reporting templates for C++ review teams +2 0 Semantic diff canonical model diff --git a/specialists/data/generated/verification_type_projects_eval.tsv b/specialists/data/generated/verification_type_projects_eval.tsv new file mode 100644 index 0000000..2ec3b9f --- /dev/null +++ b/specialists/data/generated/verification_type_projects_eval.tsv @@ -0,0 +1,9 @@ +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Plan Split monolithic endpoint into read/write paths while preserving | Split monolithic endpoint into read/write paths while preserving latency and memory budgets and updating observability d +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +3 0 Define verification and readiness surface | All core features implemented Tests passing diff --git a/specialists/data/generated/verification_type_projects_train.tsv b/specialists/data/generated/verification_type_projects_train.tsv new file mode 100644 index 0000000..f3692e1 --- /dev/null +++ b/specialists/data/generated/verification_type_projects_train.tsv @@ -0,0 +1,51 @@ +3 0 Define verification and readiness surface | Build backup retention manager with policy tiers, legal hold support, and deletion safety checks. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | Build session store service with token revocation, idle timeout, and concurrent login policy. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | All core features implemented Tests passing +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +3 0 Plan Build a streaming detector with keyed state windows | Build a streaming detector with keyed state windows, out-of-order event handling, watermark policy, and deterministic re +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | Build an inference API with model loading, request validation, and batch prediction endpoint. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | Build a whiteboard sync backend with operational ordering and conflict-safe patch application. +3 0 Define verification and readiness surface | Build job queue service with delayed jobs, retries, visibility timeout, and metrics. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | Build calendar backend with events, recurrence rules, reminders, and timezone normalization. +3 0 Define verification and readiness surface | All core features implemented Tests passing +4 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | Build a CDC replication worker that handles schema evolution, backfill merges, checkpoint consistency, and deterministic +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +4 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +2 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. diff --git a/specialists/data/generated/verification_type_train.tsv b/specialists/data/generated/verification_type_train.tsv new file mode 100644 index 0000000..e9bba34 --- /dev/null +++ b/specialists/data/generated/verification_type_train.tsv @@ -0,0 +1,978 @@ +3 0 Sprint 51 integration summary + regression +4 0 Translation Report +1 0 `whetstone_check_tool_compatibility` MCP tool +2 0 C/C++ Constructive Edit Adapter +3 0 Sprint 99 integration summary + regression +2 0 Stage checkpoint and resume model +4 0 Dynamic family acceptance report + review queue wiring +3 0 Sprint 181 Integration Summary +2 0 Semantic-noise diff suppressor model +0 0 Invariant and intent-diff checks +1 0 `whetstone_analyze_rust_semantics` MCP tool +4 0 Program constitution + handbook publication bundle +1 0 `whetstone_generate_patch_candidates` MCP tool +0 0 Unmappable semantic blocklist + mandatory review gate +1 0 Phase 15a Integration Tests +2 0 Debug policy constraint model +3 0 Sprint 177 Integration Summary +3 0 Sprint 56 integration summary + regression +0 0 Canonical fixture and oracle dataset packs +2 0 Wire `ArchitectIntakeProcessor` into `whetstone_architect_intake` handler +2 0 `whetstone_save_repro_packet` MCP tool +4 0 Guided workflow state machine for self-service runs +3 0 Sprint 64 integration summary + regression +2 0 Auto Type Deduction +1 0 `whetstone_get_governance_state` MCP tool +1 0 `whetstone_verify_scientific_migration` MCP tool +4 0 Replay and determinism report artifact +4 0 Conflict/merge qualification report artifact +4 0 Reviewer effectiveness report artifact +3 0 Sprint 189 Integration Summary +3 0 End-to-end constructive smoke scenarios per language family +2 0 `whetstone_record_attempt` MCP tool +3 0 Benchmark trend regression detector +0 0 Context Window Optimization +3 0 Sprint 191 Integration Summary +2 0 Diagnostic normalization canonical model +2 0 ABI and calling-convention canonical model +0 0 Go lowering adapter v1 +1 0 `whetstone_get_mcp_closure_status` MCP tool +2 0 Certification dashboard data model +1 0 Phase 21a Integration +3 0 Patch execution audit bundle +4 0 Permission conformance report artifact +4 0 API/ABI preservation report artifact +1 0 Apiary browser panel +3 0 Pair regression detector with severity levels +3 0 Sprint 141 integration summary + regression +2 0 Resource budget model (memory/CPU/IO) +1 0 `whetstone_plan_budget_allocation` MCP tool +4 0 Multi-language constructive loop report artifact +1 0 `whetstone_list_toolchain_providers` runtime implementation +2 0 `whetstone_capture_handoff_packet` MCP tool +1 0 `whetstone_probe_toolchain_provider` MCP tool +2 0 Orchestrator MCP handoff packet schema +3 0 Sprint 71 integration summary + regression +3 0 Sprint 55 integration summary + regression +3 0 Sprint 68 integration summary + regression +3 0 Sprint 142 integration summary + regression +3 0 Sprint 169 Integration Summary +1 0 Architect Tooling — RPC + MCP +3 0 Sprint 129 integration summary + regression +0 0 Decision consistency analyzer across reviewers +2 0 Markdown Spec Parser +0 0 Candidate generation engine from failure taxonomy +1 0 `whetstone_get_representative_language_profile` MCP tool +4 0 Web ecosystem migration report template +1 0 `whetstone_architect_intake` MCP Tool +2 0 Root-cause clustering model +4 0 Data family acceptance report +2 0 Escalation reason classifier model +0 0 Cross-platform CMake target generator +1 0 `whetstone_list_toolchain_providers` MCP tool +1 0 Sprint 37 Integration + Summary +1 0 `whetstone_select_language_adapter` MCP tool +3 0 Sprint 195 Integration Summary +0 0 Module Decomposition Engine +0 0 Governance and safety invariant revalidation pass +1 0 `whetstone_get_century_status` MCP tool +2 0 Output payload shape profiler model +1 0 Phase 23a Integration +0 0 Background task slots (parallel agent work) +3 0 Adapter health and conformance score model +0 0 Placeholder/TODO detector and policy +3 0 Phase 18b Integration + Sprint Summary +1 0 Deterministic replay pack composer +3 0 Sprint 125 integration summary + regression +4 0 Route integrity report artifact +0 0 Security scan orchestrator for generated targets +0 0 ARM Assembly Parser +3 0 Sprint 43 Integration Summary +2 0 Feedback aggregation and prioritization model +3 0 Sprint 210 Integration Summary +0 0 Floating-point determinism risk classifier +0 0 Package state probe +0 0 Window Chrome + Hierarchy Pass +1 0 `whetstone_list_language_adapters` MCP tool +3 0 Rollout health scoring model +3 0 Sprint 135 integration summary + regression +0 0 Threshold recommendation engine +4 0 Translation Report +3 0 Timing regression runner integration +1 0 `whetstone_assemble_fix_context` MCP tool +4 0 Standards publication bundle +2 0 Runtime selection decision packet model +0 0 WAT Annotation Mapping +2 0 Determinism qualification packet model +3 0 Sprint 60 integration summary + release marker +3 0 Sprint 128 integration summary + regression +4 0 Permission conformance report artifact +1 0 `whetstone_validate_patch_candidate` MCP tool +0 0 Capability Discovery Panels +4 0 Federated governance report generator +3 0 Sprint 126 integration summary + regression +4 0 Resilience operations report artifact +3 0 Sprint 144 integration summary + regression +1 0 Sprint 32 Integration + Summary +3 0 Sprint 156 integration summary + regression +0 0 Workflow Status Overlay +2 0 Distributed evidence quality scorer model +1 0 `whetstone_get_tool_deprecations` MCP tool +1 0 `whetstone_execute_rollout_or_rollback` MCP tool +4 0 Equivalence evidence bundle exporter +2 0 `ArchitectIntakeProcessor` — merge all requirement kinds +3 0 Sprint 115 integration summary + regression +2 0 Cross-language merge policy engine model +2 0 `run_pipeline` quality payload hardening +4 0 Per-target projection validation report +0 0 F# Parser +3 0 Sprint 184 Integration Summary +3 0 Sprint 208 Integration Summary +2 0 Determinism gate policy model +3 0 Sprint 120 integration summary + regression +2 0 Strict contract mode for generation +0 0 Traceability chain generator (origin->current) +1 0 `whetstone_set_authoring_mode` MCP tool +4 0 Cost Tracking + Reporting +4 0 Regulated-domain migration report template +1 0 `whetstone_reduce_repro_command` MCP tool +1 0 `whetstone_preview_regeneration_diff_generic` MCP tool +1 0 Phase 10d Integration Tests +4 0 Replay/transaction reliability report artifact +0 0 KPI delta analysis engine for block review +3 0 Sprint 182 Integration Summary +0 0 Swarm progress and fault recovery policy +0 0 Process state probe +2 0 Transpilation knowledge graph schema +4 0 Lock validation aggregate report fields +4 0 Weekly quality report artifact generator +2 0 `whetstone_preview_text_ast_merge_generic` MCP tool +3 0 Queue health scoring model +3 0 Drift audit report artifact +4 0 Evidence bundle store + deterministic ordering +0 0 Pattern applicability matcher from graph context +2 0 Conflict preview and impact estimator model +2 0 Contract breach severity classifier +0 0 Long-range benchmark corpus setup +2 0 Debug hint template model +3 0 Sprint 90 integration summary + full-program regression +0 0 Recursive object/array rule generator +4 0 HiveMind job publisher from editor +0 0 Constraint encoding and strict scalar handling +4 0 Runtime compatibility risk report generator +4 0 `whetstone_get_reference_conformance_report` MCP tool +1 0 `whetstone_build_debug_handoff` MCP tool +3 0 Sprint 91 integration summary + regression +1 0 `whetstone_list_campaign_checkpoints` MCP tool +2 0 Cross-service contract checkpoint model +0 0 Snapshot freeze workflow and integrity checks +1 0 `whetstone_export_mcp_replay_pack` MCP tool +1 0 Regional rollout profile integration +3 0 Sprint 190 Integration Summary +3 0 Phase 19b Integration + Sprint Summary +0 0 Secure-by-Default Code Generation +0 0 PostgreSQL Generator +2 0 Debug session handoff model +1 0 `whetstone_configure_hivemind` MCP tool +1 0 `whetstone_generate_safe_patch_plan` MCP tool +2 0 Build output parser — GCC/Clang format +4 0 `whetstone_start_guided_migration` MCP tool +2 0 Best-practice pack schema and versioning +4 0 Self-service adoption report artifact +1 0 `whetstone_get_language_matrix` MCP tool +0 0 Calibration corpus loader +1 0 `whetstone_score_failure_triage` MCP tool +3 0 Full regression + architecture gate +0 0 Event-order and state-sync verifier +3 0 Sprint 127 integration summary + regression +1 0 `whetstone_run_bisect_debug` MCP tool +0 0 Cross-Language Consistency Gate +1 0 `whetstone_get_execution_attestation` MCP tool +4 0 Reviewer effectiveness report artifact +3 0 Sprint 81 integration summary + regression +0 0 WAT Cross-Language Projection +0 0 Real provider probe execution paths +0 0 Next-block priority planner and constraint binder +0 0 Baseline-vs-target perf comparator +2 0 Forecast confidence and uncertainty model +4 0 Team onboarding readiness report artifact +0 0 ARM assembly lowering adapter v1 +3 0 Class-heavy regression fixture suite +1 0 Cost telemetry integration in metrics stack +4 0 `whetstone_publish_next_epoch_plan` MCP tool +1 0 `whetstone_get_polyglot_cutover_readiness` MCP tool +1 0 `whetstone_list_representative_language_profiles` MCP tool +2 0 `ABTestComparison` — compare two session records +1 0 Strategy recommendation hooks from graph context +0 0 Query behavior equivalence runner (set-based diff checks) +3 0 Sprint 70 integration summary + regression +2 0 Cross-language capability contract normalizer model +1 0 `whetstone_attach_multimodal_evidence` MCP tool +0 0 Deterministic fallback policy for unsupported operations +3 0 Sprint 174 Integration Summary +4 0 Drift outlook report artifact +3 0 Sprint 85 integration summary + regression +1 0 Tool reachability runner with deterministic probes +3 0 Phase 11e Integration + Sprint Summary +4 0 Constructive GA publication bundle artifact +4 0 Regeneration stability report artifact +4 0 Escalation report composer +2 0 Migration template schema and catalog model +4 0 Embedded migration risk report template +4 0 Self-service adoption report artifact +4 0 Compatibility/deprecation report artifact +3 0 Sprint 157 integration summary + regression +3 0 Dependency Audit Annotations +2 0 Porting cost model v1 +1 0 Phase 26a Integration +4 0 `whetstone_get_federated_verification_report` MCP tool +2 0 Campaign resume planner model +2 0 Toolchain provider interface schema +0 0 Security-Preserving Translation +3 0 Sprint 97 integration summary + regression +3 0 Rebuild `whetstone_mcp` + smoke test +4 0 Federated governance report generator +0 0 Scenario — Legacy Modernization +2 0 Constrained Routing Ruleset Extension +0 0 Architecture claim linkage engine +1 0 `whetstone_poll_iteration_job` MCP tool +4 0 Practice adoption and outcome report artifact +1 0 Phase 27a Integration +1 0 `whetstone_validate_debug_action` MCP tool +1 0 `whetstone_get_epoch_block_status` MCP tool +4 0 Evidence bundle store + deterministic ordering +2 0 Contract completeness validator +3 0 Sprint 75 integration summary + regression +2 0 External verifier adapter interface +2 0 Recursion bound/termination contracts +1 0 `whetstone_verify_architecture_consistency` MCP tool +2 0 Data-model field materialization from source AST +0 0 Route integrity gate bindings +1 0 Phase 24a Integration +2 0 Deprecation policy model for unstable/obsolete pairs +4 0 `whetstone_publish_roadmap_epoch` MCP tool +2 0 Capability decision packet model +0 0 `whetstone_restore_hybrid_checkpoint` runtime implementation +3 0 Sprint 53 integration summary + regression +3 0 Sprint 164 Integration Summary +0 0 Fine-grained capability grant engine +0 0 Queue remediation call-plan emission +4 0 Language capability conformance report artifact +1 0 Phase 10e Integration Tests +4 0 `whetstone_get_pair_scorecard` MCP tool +1 0 `whetstone_get_session_state` MCP tool +2 0 C++ constructive readiness scorer model +0 0 Capability Vocabulary & Validation +3 0 Sprint 95 integration summary + regression +2 0 Region policy overlay model +2 0 Bullet-list requirement extractor +0 0 Async mapping strategy (coroutines/task runtime profiles) +4 0 Efficiency diagnostics and trend report +1 0 Semantic validator core API +3 0 Sprint 77 integration summary + regression +0 0 Scheme Generator +3 0 Sprint 111 integration summary + regression +2 0 `whetstone_replay_repro_packet` MCP tool +4 0 Constructive GA publication bundle artifact +2 0 Debug assist packet bundle model +4 0 Tool-surface diagnostics report artifact +2 0 Argument/result canonicalizer model +4 0 Policy tuning report artifact +3 0 Sprint 197 Integration Summary +2 0 Root-cause ranking policy model +1 0 `whetstone_plan_swarm_maintenance` MCP tool +4 0 Weekly quality report artifact generator +3 0 Sprint 104 integration summary + regression +4 0 Dynamic family acceptance report + review queue wiring +1 0 `whetstone_propose_adapter_improvements` MCP tool +0 0 Blocked reason normalization +4 0 Contract conformance report artifact +0 0 Maintenance cadence scheduler +1 0 `whetstone_get_debug_campaign_status` MCP tool +3 0 Sprint 176 Integration Summary +0 0 VB.NET Generator +1 0 `whetstone_get_failure_trends` MCP tool +3 0 Sprint 145 integration summary + regression +1 0 Phase 14b Integration +2 0 Governance model refresh framework +3 0 Sprint 114 integration summary + regression +2 0 Loop retry and timeout policy model +4 0 Epoch kickoff report artifact +3 0 Session state machine transitions +2 0 Epoch-2 workstream schema and ownership map +0 0 Reviewer-focused semantic diff summarizer +3 0 Sprint 78 integration summary + regression +0 0 Cross-team benchmark comparator +0 0 Historical KPI aggregator +0 0 x86 assembly lowering adapter v1 +0 0 Async, Parallelism & Error Handling Annotations +4 0 Benchmark publication bundle +1 0 Tool manifest build-gate integration +0 0 Fix `visitModule` in Python, Rust, Go, Elisp generators +1 0 Pipeline support for injected capability signals +4 0 Requirements gap report +4 0 Release readiness scoreboard +2 0 Packet tagging integration across pipelines +3 0 Sprint 105 integration summary + regression +3 0 Sprint 108 integration summary + regression +0 0 Java lowering adapter v1 +4 0 `whetstone_get_numerical_risk_report` MCP tool +1 0 `whetstone_generate_project` MCP tool +4 0 Representative language readiness report artifact +0 0 `.whetstone.json` for the editor project itself +3 0 Sprint 46 integration summary +3 0 Audit trail and inspection +4 0 Reviewer effectiveness report artifact +4 0 Patch proposal review report artifact +3 0 Sprint 58 integration summary + regression +1 0 LTS risk watchlist and escalation hooks +3 0 Sprint 196 Integration Summary +4 0 Constructive GA publication bundle +1 0 .NET Integration +0 0 WorkerRegistry.h — Worker Abstractions +3 0 Sprint 92 integration summary + regression +1 0 `whetstone_marketplace_search` MCP tool +3 0 Full Test Suite Regression +2 0 Class-generation contract schema +4 0 Program constitution + handbook publication bundle +1 0 `whetstone_transpile_query_family` MCP tool +3 0 Sprint 72 integration summary + regression +1 0 `whetstone_export_repro_jsonl` MCP tool +0 0 Production summary hardening +0 0 Quarterly lock/review cadence policy +2 0 Cross-language regeneration quality packet model +2 0 Requirement Normalization and Conflict Detection +4 0 Compatibility/deprecation report artifact +0 0 `SelfContainmentScorer` — score a taskitem 0–100 +0 0 Ownership mapping policy engine (`unique_ptr/shared_ptr/value`) +4 0 Incremental sync report artifact +2 0 Wiring closure severity classifier +4 0 Representative language readiness report artifact +2 0 Recovery safety packet model +2 0 `whetstone_get_constructive_rollout_status` runtime implementation +3 0 Sprint 152 integration summary + regression +4 0 Semantic memory report artifact +0 0 Dependency Graph Visualization +0 0 F# lowering/raising adapters +4 0 `whetstone_get_numerical_risk_report` MCP tool +2 0 Generic incremental edit packet schema +0 0 Workflow Sidecar Persistence +0 0 Key Symbol Rendering +4 0 Multi-language constructive loop report artifact +2 0 Call Stack and Frame Inspector +1 0 Phase 20a Integration +4 0 Constructive runtime readiness report bundle +2 0 `whetstone_run_constructive_step`/`status`/`loop` runtime packet model +4 0 Contract conformance report artifact +4 0 Regeneration stability report artifact +0 0 Safety case completeness checker +4 0 Benchmark scorecard generator per pair +0 0 Ruby lowering adapter v1 (meta-programming boundaries) +3 0 Cost vs Quality Regression Suite +2 0 Resume and rollback policy runtime model +1 0 Priority-focused review queue shaping engine +0 0 Self-Hosting Test Harness +0 0 Scaffold File Generation +1 0 Phase 18a Integration +1 0 Build iteration session integration +3 0 Profile compliance auditor integration +1 0 Corpus ingestion tooling and normalization +0 0 Safety-critical deployment dossier artifact +4 0 Swarm maintenance report artifact +2 0 MCP compatibility policy schema +2 0 Stewardship and ownership policy model +2 0 Serialization + Dispatch for New Nodes +4 0 `whetstone_publish_roadmap_epoch` MCP tool +3 0 Sprint 153 integration summary + regression +3 0 Sprint 54 integration summary + regression +2 0 Patch risk labeler model +1 0 `whetstone_resume_debug_campaign` MCP tool +4 0 Reliability operations report artifact +2 0 `whetstone_rollback_constructive_transaction` MCP tool +0 0 CMake generator from Whetstone project description +3 0 Intake regression corpus loader +2 0 Debug recipe step model +4 0 Efficiency diagnostics and trend report +0 0 Completion Overlay UX Hardening +3 0 Sprint 183 Integration Summary +2 0 Execution hint extraction from requirements +2 0 Confidence synthesis model +3 0 Sprint 161 Integration Summary +0 0 Hint feature extraction from decision ledgers +3 0 Sprint 171 Integration Summary +0 0 Fuzz differential runner with seed replay +1 0 `whetstone_generate_debug_hints` MCP tool +3 0 Sprint 121 integration summary + regression +1 0 `whetstone_list_plugins` MCP tool +4 0 Benchmark publication bundle +0 0 Scope & Namespace Annotations +4 0 Campaign report bundle generator +1 0 `whetstone_get_upgrade_queue` MCP tool +3 0 Sprint 179 Integration Summary +1 0 Cross-session context bridge +4 0 Documentation + Operator Playbooks +3 0 Sprint 80 integration summary + full-program regression +2 0 Cross-tenant adapter sharing contract +4 0 Cost-vs-quality tradeoff report generator +3 0 Audit trail completeness checker +2 0 Scheduler lock contract emitter +1 0 Review SLA policy integration +1 0 Approval workflow integration with governance ledger +1 0 Static registry scanner for `register*Tools` blocks +1 0 Phase 22a Integration +1 0 `whetstone_set_runtime_profile` MCP tool +2 0 Security Annotation Types — Subject 10 +0 0 Tier promotion engine (beta->stable) +2 0 Locals + Watches Panel Data Model +1 0 `whetstone_get_related_migration_patterns` MCP tool +3 0 Phase 24b Integration + Sprint Summary +2 0 Debug stop-reason classifier model +1 0 Modernization RPC + MCP +3 0 Sprint 194 Integration Summary +2 0 Architect Review Interface +3 0 Sprint 175 Integration Summary +1 0 `whetstone_register_external_verifier` MCP tool +4 0 Swarm maintenance report artifact +1 0 `whetstone_capture_distributed_failure` MCP tool +0 0 `TokenBudgetEnforcer` — trim assembled context to budget +4 0 Plugin trust and risk report artifact +3 0 Sprint 93 integration summary + regression +1 0 New Languages Integration + MCP +2 0 Pipeline promotion packet integration +2 0 Real GA evidence packet generation +0 0 Gate-driven remediation planner +1 0 Migration Execution Integration +3 0 Hint safety guardrails and audit fields +4 0 Interop publication bundle +4 0 Portfolio strategy report artifact +1 0 `whetstone_set_language_rollout_tier` MCP tool +2 0 KPI anomaly triage model +0 0 Config file probe +0 0 C++ runner adapter for harness +1 0 `whetstone_explain_transpilation_decision` MCP tool +4 0 Federated evidence bundle composer +3 0 `whetstone_regression_guard` MCP tool +1 0 Skill progression tracker integration +0 0 Validation capability-signal scoring/classification +4 0 `whetstone_publish_next_epoch_plan` MCP tool +1 0 `whetstone_plan_tool_migrations` MCP tool +1 0 `whetstone_get_debug_checklist` MCP tool +2 0 `AgentSessionRecorder` — per-session tool call instrumentation +2 0 Workspace-source freshness comparator model +0 0 Queue+validation metadata enrichment +1 0 `whetstone_get_debug_metrics` MCP tool +0 0 Architecture Templates +3 0 Sprint 185 Integration Summary +4 0 Replay and determinism report artifact +2 0 STL algorithm lifting from IR intent tags +3 0 Epoch block health re-baseline model +0 0 Alert policy engine for SLO breaches +2 0 Validation lock name-format checks +1 0 `whetstone_generate_inference_job` MCP tool +0 0 Smart Pointer Patterns +0 0 Legacy semantic recovery graph engine +0 0 Context Bundle Minimizer +3 0 Sprint 47 integration summary + regression +2 0 Patch execution record model +3 0 Sprint 137 integration summary + regression +3 0 Sprint 163 Integration Summary +1 0 Phase 10c Integration Tests +2 0 Debugging metrics packet (`time_to_green`, `iterations`, etc.) +1 0 `whetstone_export_tool_manifest` MCP tool +1 0 `whetstone_run_continuity_drill` MCP tool +1 0 `whetstone_get_debug_trace` MCP tool +0 0 `PrerequisiteOpResolver` — resolve ops to workspace paths +1 0 Phase 19a Integration +3 0 Sprint 124 integration summary + regression +2 0 `whetstone_get_safety_profile_requirements` MCP tool +1 0 Generator Updates + Phase 11c Integration +3 0 Sprint 118 integration summary + regression +0 0 False-green diagnostic class +2 0 Sidecar AST Persistence +1 0 `whetstone_queue_ready` MCP Tool +0 0 Conflict avoidance and lock orchestration engine +3 0 Sprint 136 integration summary + regression +3 0 Sprint 61 integration summary + regression +4 0 Plugin trust and risk report artifact +2 0 Bisect candidate selector model +1 0 `whetstone_step_debug_campaign` MCP tool +4 0 Runtime execution trace report artifact for sync/regeneration/merge tools +4 0 Language capability conformance report artifact +1 0 Phase 14d Integration + Sprint 14 Summary +1 0 Phase 11b Integration Tests +3 0 Transition plan model for Sprint 101+ workstreams +4 0 Cost-vs-quality tradeoff report generator +1 0 Sprint 26 Integration + Summary +2 0 Swarm maintenance job decomposition model +2 0 Fast rollback orchestration contract +0 0 Continuity drill framework and scoring +0 0 Multi-language minimal test harness runner +1 0 `whetstone_list_distributed_failures` MCP tool +0 0 Top-level strictness gate +4 0 Portfolio strategy report artifact +3 0 Century milestone publication package +3 0 Sprint 203 Integration Summary +3 0 Sprint 41 Integration Summary +0 0 Failure fixture catalog for deterministic debug benchmarks +4 0 Debt burn-down report artifact +2 0 Minimal repro reducer model +2 0 Result Acceptance Protocol +2 0 Host boundary + FFI contract generator +0 0 Workspace Onboarding Flow +4 0 Generic sync determinism report artifact +3 0 Post-incident continuity audit report model +2 0 SLM debug readiness scoring model +2 0 `whetstone_run_constructive_ga_gate` runtime implementation +4 0 HiveMind job publisher from editor +3 0 Docking Reliability Audit + Deterministic Layout Rules +2 0 Pair-specific known-limitations generator +0 0 Compliance policy binding engine +0 0 Context Assembly + Budget +0 0 Artifact fingerprinting and drift detection +3 0 Sprint 131 integration summary + regression +1 0 Verification gate integration for architecture artifacts +0 0 Python + Java Parser Deepening +3 0 Sprint 202 Integration Summary +2 0 End-to-end validation with the failing A/B test spec +2 0 Explainability compression model for rapid review +0 0 Scheme lowering/raising adapters +2 0 Failure trend tracker model +2 0 Range-Based For + Structured Bindings +3 0 Sprint 165 Integration Summary +4 0 Marketplace governance report artifact +3 0 Sprint 192 Integration Summary +1 0 `whetstone_start_debug_campaign` MCP tool +0 0 Property-based equivalence runner +2 0 Rollback ledger model +2 0 Capability negotiation and fallback policy model +0 0 Trial evaluation rubric (quality/risk/cost) +1 0 `whetstone_plan_deprecation_or_upgrade` MCP tool +0 0 Constraint completeness diagnostics +4 0 `whetstone_get_artifact_trust_report` MCP tool +0 0 Find and Replace Improvements +2 0 Compatibility simulation runner model +1 0 Binary/footprint gate integration +2 0 `whetstone_validate_tool_contracts` MCP tool +2 0 JavaScript lowering adapter v2 (prototype/object model) +2 0 Rust/Go Constructive Edit Adapter +1 0 Phase 25b Integration +0 0 C Generator +1 0 `whetstone_propose_policy_tuning` MCP tool +3 0 Scope and Milestone Decomposer +1 0 Policy-compliance auto-checkpoint integration +0 0 Incident containment automation for compromised components +2 0 Worker Efficiency Dashboard Data Model +4 0 `whetstone_get_artifact_trust_report` MCP tool +3 0 Sprint 187 Integration Summary +3 0 Sprint 123 integration summary + regression +3 0 Sprint 146 integration summary + regression +2 0 Constructive transaction schema +2 0 Representative language conformance packet model +0 0 Projection metadata summary export +4 0 Data family acceptance report +4 0 Documentation + Operator Playbooks +1 0 `whetstone_validate_patch_proposal` MCP tool +1 0 `whetstone_probe_tool_reachability` MCP tool +2 0 Hybrid core contract schema v1 +2 0 IR lowering interface and adapter API +1 0 Adoption telemetry integration +1 0 `whetstone_select_mcp_runtime` MCP tool +3 0 Sprint 113 integration summary + regression +1 0 `whetstone_plan_debt_burndown` MCP tool +4 0 Self-Hosting Progress Report + Phase Integration +3 0 Tier transition guardrail and rollback runtime model +1 0 `whetstone_save_campaign_checkpoint` MCP tool +2 0 `whetstone_run_spec_conformance` MCP tool +0 0 Kotlin lowering/raising adapters +1 0 `whetstone_dry_run_patch` MCP tool +4 0 Tool-surface diagnostics report artifact +2 0 `whetstone_get_pair_stability_forecast` MCP tool +4 0 Misclassification report export +1 0 Phase 38a Integration +3 0 Full historical regression + post-release checklist +0 0 Actor-to-thread/async projection policy set +2 0 Strict-mode escalation for low-specificity tasks +0 0 Preventive mitigation planner +2 0 Step Spec Expander (tests: unit, negative, boundary) +0 0 Optimization Annotation Completion +0 0 Parse AnnotationValidatorExtended.h +4 0 `whetstone_get_sync_identity_report` MCP tool +0 0 Value-Forward Onboarding Flow +3 0 Runtime health probe runner +3 0 Sprint 44 Integration Summary +0 0 Federated reproducibility validator +3 0 Sprint 89 integration summary + regression +4 0 `whetstone_start_guided_migration` MCP tool +2 0 Tool-surface checksum packet model +3 0 Sprint 211 Integration Summary +1 0 `whetstone_list_migration_templates` MCP tool +0 0 Deterministic blocker taxonomy +2 0 Language-neutral operation envelope model +2 0 `whetstone_record_debug_trace` MCP tool +1 0 `whetstone_export_debug_campaign_bundle` MCP tool +1 0 `whetstone_run_test_iteration` MCP tool +4 0 Hybrid mode readiness report artifact +3 0 Sprint 140 integration summary + full-program regression +0 0 Agent task status overlay +1 0 Workstream drift detector integration +4 0 Replay/transaction reliability report artifact +1 0 Annotation RPC Methods +4 0 Embedded migration risk report template +4 0 Explainability report template pack +1 0 `whetstone_install_or_disable_plugin` MCP tool +0 0 Modernization Suggestions +2 0 Rollout blast-radius estimator +4 0 Handoff integrity report artifact +1 0 `whetstone_get_runtime_fingerprint` MCP tool +0 0 Inline Annotation Badges +4 0 Web ecosystem migration report template +0 0 T-SQL lowering/raising adapters +1 0 Tool call visualization +2 0 Ecosystem drift signal model +4 0 Conflict/merge report artifact +2 0 Maintenance TCO forecasting engine +4 0 Epoch renewal publication bundle +0 0 AppImage / .deb package generator +3 0 Sprint 52 integration summary + regression +3 0 Sprint 122 integration summary + regression +4 0 Program benchmarking report bundle +3 0 Sprint 110 integration summary + full-program regression +1 0 `whetstone_start_onboarding_path` MCP tool +1 0 `whetstone_transpile_systems_family` MCP tool +0 0 ProjectionGenerator — Subject 5-8 + Semantic Visitors +1 0 `whetstone_get_api_semantics` MCP tool +1 0 `whetstone_optimize_review_queue` MCP tool +0 0 Cross-language class emission parity tests +4 0 Toolchain readiness report artifact +1 0 `whetstone_validate_policy_tuning` MCP tool +4 0 MCP Tool Documentation +0 0 Queue/validation fingerprint convergence +3 0 `TaskitemQualityAuditor` — batch report +2 0 Runtime selection policy model +3 0 Phase 23b Integration + Sprint Summary +3 0 Strictness regression suite +3 0 Sprint 48 integration summary + regression +1 0 `whetstone_plan_polyglot_migration` MCP tool +2 0 SQL canonical query IR layer +4 0 C++ constructive loop report artifact +3 0 Sprint 151 integration summary + regression +1 0 `whetstone_generate_safety_case` MCP tool +4 0 Runtime execution trace report artifact for sync/regeneration/merge tools +4 0 Security hardening report artifact +4 0 Public spec for verification evidence bundles +3 0 Sprint 158 integration summary + regression +0 0 KPI-to-workstream binding engine +0 0 End-to-end replay + rollback scenario suite +1 0 Transpilation RPC + MCP +1 0 `whetstone_verify_handoff_integrity` MCP tool +3 0 Phase 17b Integration + Sprint Summary +4 0 Constructive runtime readiness report bundle +0 0 Target projection blocker taxonomy +1 0 Runtime `tools/list` snapshot normalizer +0 0 C# Generator +4 0 MCP Tool Documentation +4 0 Generic sync determinism report artifact +2 0 AST mutation preview +0 0 Config auto-discovery (walk up from CWD) +4 0 SemannoFormat.h — Comment Format Standard +3 0 Sprint 193 Integration Summary +1 0 Test iteration session integration +3 0 Century milestone health scoreboard model +2 0 Allowed-file scope policy model +3 0 Sprint 59 integration summary + regression +2 0 Debug recipe library model +1 0 Real sync tool binding +4 0 Guided workflow state machine for self-service runs +2 0 Compatibility risk packet model +1 0 Phase 28a Integration +3 0 Sprint 57 integration summary + regression +2 0 Candidate ranking and recommendation model +4 0 Regulated-domain migration report template +0 0 Union support (`oneOf`/`anyOf`/`allOf`) +4 0 Reliability operations report artifact +2 0 `whetstone_export_tool_contracts` MCP tool +3 0 Sprint 49 integration summary + regression +1 0 MCP replay diagnostics artifact +0 0 Tenant isolation guardrails for logs/artifacts +1 0 `whetstone_suggest_test_fix` MCP tool +1 0 `whetstone_run_mcp_closure_gate` MCP tool +3 0 Sprint 168 Integration Summary +2 0 Pair stability forecasting engine +3 0 Sprint 162 Integration Summary +3 0 Sprint 206 Integration Summary +2 0 Runtime-backed constructive release certification packet model +0 0 Full loop driver (end-to-end stepping) +1 0 Parallel dispatch optimizer with dependency constraints +0 0 Queue lock-conflict detection +2 0 Debug campaign spec model +2 0 Representative language toolchain provider registry model +1 0 `whetstone_run_interop_testbed` MCP tool +3 0 Sprint 73 integration summary + regression +2 0 Concurrency Annotations — Primitives & Memory Model +2 0 Job schema → typed C++ structs generator +4 0 LTS publication bundle + operator handbook +2 0 Hint confidence + uncertainty packet format +0 0 Workspace mode policy bindings +0 0 Notification + Status Signaling Refresh +1 0 `whetstone_get_portfolio_economics` MCP tool +4 0 Setup verification report +3 0 Sprint 76 integration summary + regression +1 0 `whetstone_start_feedback_loop` MCP tool +2 0 Non-C++ adapter conformance harness model +1 0 `whetstone_generate_setup_script` MCP tool +0 0 Java raising adapter v1 +2 0 EnvironmentSpec Schema & AST Node +0 0 Multi-language compile gate executor +1 0 `whetstone_normalize_diagnostics` MCP tool +2 0 Stable include/order regeneration model +0 0 Queue lock conflict summary metrics +3 0 Sprint 199 Integration Summary +3 0 Sprint 204 Integration Summary +3 0 Sprint 119 integration summary + regression +0 0 Test failure classifier +4 0 Setup verification report +4 0 Runtime selection report artifact +1 0 `whetstone_run_build_iteration` MCP tool +4 0 Reliability and rollback report artifact +3 0 Sprint 209 Integration Summary +3 0 Sprint 170 Integration Summary +3 0 Regression watchlist for promoted pairs +4 0 Documentation pack for language support + migration playbooks +2 0 AST-to-text regeneration policy schema +2 0 Nullability + optionality canonical model bridge +1 0 Corpus version diff and migration tools +0 0 Artifact quality scoring and ranking engine +0 0 Kotlin Generator +2 0 Autonomous loop controller with “green or explicit blocked” contract +1 0 `whetstone_validate_patch_scope` MCP tool +3 0 Sprint 198 Integration Summary +3 0 Sprint 84 integration summary + regression +2 0 Source-vs-runtime drift comparator model +1 0 `whetstone_generate_reprojection_plan` MCP tool +2 0 Localization-ready artifact schema +0 0 Release Readiness Gate Pack +1 0 `whetstone_apply_best_practice_pack` MCP tool +0 0 Go + C++ + Elisp Parser Deepening +4 0 Self-Hosting Progress Report + Phase Integration +2 0 `whetstone_ingest_legacy_to_ir` MCP tool +3 0 Sprint 74 integration summary + regression +0 0 Annotate Whetstone's Own Code +0 0 C interop adapter deepening +2 0 Refactor pattern schema and metadata model +2 0 Python/TypeScript Constructive Edit Adapter +3 0 Sprint 159 integration summary + regression +0 0 Scope ambiguity diagnostics +4 0 Standards publication bundle +1 0 `whetstone_set_zero_trust_policy` MCP tool +0 0 Benchmark runner and metrics export +0 0 Template parameter validation engine +0 0 Transpile Whetstone Modules +1 0 Phase 13c Integration — Keyboard Shortcuts Help Panel +0 0 Deterministic plan fingerprinting +3 0 Numerical regression gate integration +3 0 Sprint 147 integration summary + regression +2 0 Pair-specific hint model interface +2 0 Parallel partitioning semantics model +2 0 Node identity preservation policy model +3 0 Sprint 83 integration summary + regression +4 0 `whetstone_generate_localized_migration_report` MCP tool +2 0 Determinism and recovery qualification packet model +0 0 Starvation prevention mechanism +3 0 Sprint 166 Integration Summary +0 0 C# Parser +4 0 Guided transpilation exercise framework +3 0 Sprint 42 Integration Summary +4 0 Runtime compatibility risk report generator +2 0 Interrupt and ISR semantics packet model +1 0 Loop integration for auto-fix attempt +4 0 Marketplace governance report artifact +2 0 `whetstone_get_language_contract` MCP tool +2 0 Triage packet bundle model +1 0 `whetstone_probe_toolchain_provider` runtime implementation +4 0 Reporting templates for C++ review teams +1 0 `whetstone_query_transpile_graph` MCP tool +3 0 Sprint 86 integration summary + regression +0 0 Preconditions/postconditions validator for patterns +0 0 File Tree Panel +4 0 Runtime selection report artifact +0 0 Taskitem Generator v2 +4 0 Toolchain readiness report artifact +0 0 Promotion policy for tuned packs +4 0 Family acceptance and divergence report +0 0 Edge Case Cleanup +0 0 Promotion gate for accepted proposals +4 0 Release readiness scoreboard +3 0 Sprint 180 Integration Summary +0 0 Interop certification policy bindings +2 0 Threat Model Integration +1 0 `whetstone_analyze_data_pipeline_semantics` MCP tool +4 0 Low-level acceptance report + waiver system +3 0 Sprint 101 integration summary + regression +2 0 MCP initialize — add `instructions` field +4 0 Regeneration stability report artifact +0 0 Sprint 25 Summary + Post-25 Readiness +0 0 Trend and recurrence detector +3 0 Sprint 155 integration summary + full-program regression +1 0 Sprint 30 Integration + Summary +2 0 Environment-Aware Pipeline Hooks +2 0 Benchmark ingestion support for projection schema +0 0 Per-section recall/precision metrics +1 0 `whetstone_assemble_context` MCP tool +1 0 `whetstone_debug_until_green` MCP orchestrator tool +4 0 Per-target projection validation report +0 0 Queue blocker gap classifier +1 0 Plugin sandbox execution policy +0 0 Pattern recurrence detector from graph signals +2 0 Pair selection strategy engine (hot pairs, random sampling, full sweeps) +0 0 Fix candidate generator +3 0 Rebuild whetstone_mcp + smoke test +4 0 Explainability report template pack +3 0 `whetstone_probe_mcp_runtime_health` MCP tool +4 0 Family-level acceptance report generator +3 0 Sprint 100 integration summary + full-program regression +3 0 Sprint 98 integration summary + regression +0 0 WorkflowState.h — Project-Level Workflow Tracking +3 0 Sprint 205 Integration Summary +1 0 `whetstone_get_epoch_workstreams` MCP tool +2 0 Prioritization scoring model (impact x frequency x tier) +4 0 Lock metadata documentation packet +4 0 Escalation report composer +3 0 Sprint 160 integration summary + full-program regression +2 0 Capability declaration struct generator +4 0 Lock validation aggregate report fields +3 0 Sprint 167 Integration Summary +1 0 Gold benchmark curation pipeline +1 0 RoutingEngine.h — Annotation-to-Dispatch Logic +4 0 `whetstone_get_federated_verification_report` MCP tool +2 0 Generic/monomorphization intent model +1 0 `whetstone_estimate_porting_cost` MCP tool +4 0 C++ constructive loop report artifact +4 0 Epoch kickoff report artifact +4 0 Cost Tracking + Reporting +0 0 Long-range blocker taxonomy +4 0 Campaign report bundle generator +4 0 `whetstone_get_sync_identity_report` MCP tool +3 0 Sprint 45 Integration Summary +0 0 Template Class Declarations +1 0 Route-to-deterministic policy hook +3 0 Sprint 103 integration summary + regression +1 0 Pipeline summary propagation of call plan +4 0 Patch proposal review report artifact +2 0 Profile-backed readiness and qualification packet model +4 0 `whetstone_generate_localized_migration_report` MCP tool +2 0 Type System Translation +1 0 `whetstone_get_regeneration_trace_generic` MCP tool +4 0 Benchmark scorecard generator per pair +4 0 Runtime freshness report artifact +2 0 Merge impact and provenance model +2 0 New AST Nodes — Class/Interface/Generic +4 0 Family acceptance and divergence report +3 0 Sprint 130 integration summary + regression +2 0 Perf benchmark harness contract +2 0 Data lineage canonical model +3 0 Sprint 109 integration summary + regression +4 0 Security Report Generation +3 0 Safety Audit via Annotations +0 0 Candidate auto-selector +4 0 Practice adoption and outcome report artifact +1 0 Sanitizer gate integration (ASan/UBSan profile hooks) +4 0 Enterprise rollout report template +3 0 Convention Extractor + Validator (tests: unit, negative, regression) +4 0 Incremental sync report artifact +1 0 Auto-containment strategy hooks +0 0 Multi-Language Project Orchestration +1 0 Phase 12a Integration Tests +2 0 Integrity attestation packet schema +3 0 Sprint 107 integration summary + regression +2 0 Plugin manifest and capability declaration model +2 0 Session-level mode override model +3 0 Sprint 200 Integration Summary +2 0 `whetstone_run_constructive_ga_gate` MCP tool +4 0 Team onboarding readiness report artifact +1 0 Orchestrator RPC + MCP +3 0 Sprint 67 integration summary + regression +0 0 Job status subscriber panel +1 0 `whetstone_verify_simulation_port` MCP tool +2 0 Zero-trust policy model for execution surfaces +1 0 `whetstone_get_slm_debug_readiness` MCP tool +3 0 Sprint 102 integration summary + regression +4 0 Cross-language constructive execution report artifact +0 0 MySQL lowering/raising adapters +0 0 Accessibility Baseline Pass +1 0 `whetstone_get_recovery_advice` MCP tool +3 0 Sprint 106 integration summary + regression +3 0 Sprint 143 integration summary + regression +3 0 Sprint 133 integration summary + regression +4 0 Replay/transaction reliability report artifact +2 0 `whetstone_start_recording` + `whetstone_get_metrics` MCP tools +0 0 C++ Generator — Inheritance + Templates +2 0 `whetstone_preview_text_ast_merge` MCP tool +1 0 tools.json refresh — add all Sprint 32-35 tools +3 0 Sprint 65 integration summary + regression +0 0 Generator Implementations — Python/C++/Rust/Go +0 0 Multi-file conflict detection during execution +1 0 `whetstone_start_iteration_session` MCP tool +3 0 Sprint 50 integration summary + regression +1 0 `whetstone_plan_preventive_maintenance` MCP tool +3 0 Sprint 63 integration summary + regression +0 0 Pre-Apply Validation Gate +4 0 Epoch renewal publication bundle +3 0 Sprint 173 Integration Summary +4 0 Guided Architect-to-Execution Demo Mode +1 0 Semanno Sidecar Integration +4 0 Resilience operations report artifact +2 0 Contract conformance score model +4 0 Semantic memory report artifact +3 0 Sprint 134 integration summary + regression +4 0 Security Report Generation +3 0 Sprint 188 Integration Summary +4 0 Requirements gap report +2 0 C++ formatting/styling guard model +0 0 Security Testing Skeleton Generation +3 0 Sprint 82 integration summary + regression +4 0 `whetstone_get_reference_conformance_report` MCP tool +0 0 Multi-service rollback choreography policy +2 0 Test output parser — gtest format +2 0 Language operation support policy model (`edit`, `sync`, `regenerate`, `merge`, `build_test`) +1 0 `whetstone_get_certification_status` MCP tool +0 0 Deterministic threshold profile selection +3 0 Phase 21b Integration + Sprint Summary +2 0 Real replay suite wiring to runtime traces +0 0 Panel Registration System +2 0 LTS baseline criteria schema +3 0 Mode transition event packet model +4 0 Pattern efficacy report artifact +2 0 Campaign queue policy model +3 0 Sprint 149 integration summary + regression +1 0 Lint executor integration +0 0 Task Detail View +0 0 Distributed path in FeedbackLoopOrchestrator +0 0 Recommendation engine for applicable packs +4 0 Localized report generation pipeline +2 0 Agent Chat Panel model +2 0 Binary/interface diff engine integration +4 0 Constructive GA publication bundle +1 0 `whetstone_run_improvement_trial` MCP tool +2 0 Determinism mode packet (`strict`, `best-effort`) +4 0 Debt burn-down report artifact +4 0 `whetstone_get_pair_scorecard` MCP tool +0 0 Symbol Scope Extractor +2 0 MCP execution event schema +1 0 API Boundary Preservation +1 0 `whetstone_transpile_low_level_family` MCP tool +2 0 Constructive loop orchestration schema +1 0 MCP Server Configuration + Discovery +4 0 Program constitution + handbook publication bundle +1 0 Operation Selector API +4 0 Self-Hosting Metrics Report +1 0 Phase 13d Integration + Sprint 13 Summary +4 0 Localized report generation pipeline +4 0 SemannoFormat.h — Comment Format Standard +3 0 Sprint 69 integration summary + regression +4 0 Misclassification report export +0 0 Deterministic data-sample replay runner +0 0 Analytics migration dossier template +0 0 Checkpoint and rollback node insertion +0 0 Method completion planner +4 0 Regeneration stability report artifact +0 0 Dynamic strictness policy engine (`lenient/balanced/strict`) diff --git a/specialists/data/generated/worker_type_eval.tsv b/specialists/data/generated/worker_type_eval.tsv new file mode 100644 index 0000000..7dd5ef3 --- /dev/null +++ b/specialists/data/generated/worker_type_eval.tsv @@ -0,0 +1,52 @@ +0 0 Snapshot persistence (sidecar integration) +1 0 `whetstone_preview_regeneration_diff_generic` MCP tool +2 0 Phase 25a Integration +3 0 Sprint 116 integration summary + regression +0 0 Debug assist packet bundle model +1 0 `whetstone_score_failure_triage` MCP tool +3 0 Regression watchlist for promoted pairs +2 0 Architecture Templates +3 0 Phase 10d Integration Tests +2 0 Phase 21a Integration +2 0 Result Acceptance Protocol +2 0 Guided transpilation exercise framework +0 0 Patch proposal schema and provenance model +2 0 `whetstone_publish_roadmap_epoch` MCP tool +0 0 Compliance policy binding engine +2 0 Workflow Session Protocol +3 0 Sprint 181 Integration Summary +2 0 Language-specific method body scaffolding for core patterns +0 0 Sprint 27 Integration + Summary +3 0 Phase 15b Integration — Full Protocol Test +0 0 Long-range edit graph planner +3 0 Sprint 120 integration summary + regression +0 0 Federated governance report generator +3 0 Sprint 66 integration summary + regression +3 0 Sprint 204 Integration Summary +2 0 Phase 27a Integration +1 0 Review Gates +3 0 Sprint 55 integration summary + regression +0 0 LTS baseline criteria schema +0 0 Replay comparison and divergence classification runtime model +3 0 Sprint 101 integration summary + regression +2 0 Retry/Escalation Protocol for Constraint Failures +0 0 Compatibility policy profiles by interface criticality +0 0 Language support tiers + gates +2 0 Phase 13d Integration + Sprint 13 Summary +1 0 Deterministic triage queue planner model +1 0 Unmappable semantic blocklist + mandatory review gate +2 0 Phase 23a Integration +1 0 `whetstone_preview_text_ast_merge_generic` MCP tool +3 0 Sprint 132 integration summary + regression +2 0 Phase 27a Integration +3 0 Sprint 71 integration summary + regression +2 0 Phase 14b Integration +0 0 Epoch-2 workstream schema and ownership map +3 0 Sprint 141 integration summary + regression +3 0 AST-native projection benchmark suite +0 0 Certification evidence archive format v2 +2 0 Phase 29a Integration +0 0 Tier promotion engine (beta->stable) +1 0 Quarterly lock/review cadence policy +1 0 Diagnostic normalization runtime implementation model +2 0 Phase 22a Integration diff --git a/specialists/data/generated/worker_type_projects_eval.tsv b/specialists/data/generated/worker_type_projects_eval.tsv new file mode 100644 index 0000000..f254759 --- /dev/null +++ b/specialists/data/generated/worker_type_projects_eval.tsv @@ -0,0 +1,63 @@ +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Plan Build a streaming join worker correlating click and | Build a streaming join worker correlating click and conversion events with watermark logic. +0 0 Define verification and readiness surface | Build a live notifications hub with subscription filters, fanout, and reconnect replay support. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build a secrets rotation scheduler with lease tracking, retry plans, and notifications. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Define verification and readiness surface | Build kanban API with boards, columns, cards, drag reorder, and activity history. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +0 0 Plan Build an IoT telemetry gateway supporting MQTT ingestion | Build an IoT telemetry gateway supporting MQTT ingestion, validation, and stream forwarding. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a CLI todo manager with add/list/done/delete commands and JSON file storage. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a whiteboard sync backend with operational ordering and conflict-safe patch application. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +0 0 Define verification and readiness surface | Build an image conversion CLI with resize, format conversion, and batch mode support. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Plan Split monolithic endpoint into read/write paths while preserving | Split monolithic endpoint into read/write paths while preserving latency and memory budgets and updating observability d +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +3 0 Define verification and readiness surface | Build portfolio tracker with holdings, transactions, PnL snapshots, and benchmark comparison. +0 0 Define verification and readiness surface | Build checkout backend coordinating address, shipping options, totals, and payment intent creation. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +0 0 Define verification and readiness surface | Build chat bot backend with slash commands, permission checks, and scheduled digests. +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build a contacts service with import/export, deduplication, and fuzzy name search. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +0 0 Define verification and readiness surface | Build realtime polling backend with vote dedupe, room join, and result broadcast. +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +0 0 Plan Upgrade API contract from v1 to v2 backend | Upgrade API contract from v1 to v2: backend handlers, OpenAPI schema, generated client SDK, frontend integration, and co +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +0 0 Plan Build a streaming detector with keyed state windows | Build a streaming detector with keyed state windows, out-of-order event handling, watermark policy, and deterministic re +0 0 Define verification and readiness surface | Build a release notes generator from commits, PR labels, and issue references. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Define verification and readiness surface | Build a change-data-capture processor that applies upserts/deletes and checkpoints offsets. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +0 0 Define verification and readiness surface | Build a model registry API supporting stages, approvals, rollback, and signed artifact links. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua diff --git a/specialists/data/generated/worker_type_projects_train.tsv b/specialists/data/generated/worker_type_projects_train.tsv new file mode 100644 index 0000000..eab20e0 --- /dev/null +++ b/specialists/data/generated/worker_type_projects_train.tsv @@ -0,0 +1,361 @@ +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build job queue service with delayed jobs, retries, visibility timeout, and metrics. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +0 0 Define verification and readiness surface | Build a CLI password vault with encrypted local storage and master-key unlock. +0 0 Define verification and readiness surface | Build order management with status machine, cancellation rules, and shipment events. +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build a registry mirror service with digest verification, cache pruning, and pull-through. +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +0 0 Define verification and readiness surface | Build a websocket chat server with rooms, presence, typing indicators, and message history. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +0 0 Define verification and readiness surface | Build subscription billing engine with proration, trials, coupons, and failed-payment retries. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +0 0 Define verification and readiness surface | Build invoice generator with tax calculation, due schedules, and PDF export metadata. +0 0 Define verification and readiness surface | Build a webhook dispatcher with retries, signature verification, dead-letter queue, and backoff. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Plan Build a ticker stream service with topic subscriptions | Build a ticker stream service with topic subscriptions and throttled update delivery. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a CLI log tailer with multi-file follow, regex filters, and output highlighting. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a CSV analyzer CLI with schema inference, summary stats, and filter expressions. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build returns API with RMA creation, restock policy, and refund eligibility checks. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a service monitor that runs health checks, tracks SLO windows, and pages on breaches. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build expense backend with receipt OCR placeholders, category budgets, and recurring expenses. +0 0 Define verification and readiness surface | Build a harness that transpiles class-heavy source modules across multiple target languages and validates class/method p +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +0 0 Define verification and readiness surface | Build matchmaking service with queue buckets, skill-based pairing, and timeout fallback. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +0 0 Plan Build an incremental snapshot builder that compacts event | Build an incremental snapshot builder that compacts event logs into query-optimized state. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +0 0 Define verification and readiness surface | Build a REST Todo API with CRUD, pagination, filtering by status, and SQLite persistence. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Plan Add normalized address schema migrate legacy rows with | Add normalized address schema, migrate legacy rows with backfill job, dual-read/write transition, and rollback script. +0 0 Define verification and readiness surface | Build a log enrichment batch job joining geo data and user agents with partitioned output. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Define verification and readiness surface | Build a URL shortener API with custom aliases, redirect tracking, and expiration support. +0 0 Define verification and readiness surface | Build meeting notes API with templates, attendee action items, and follow-up reminders. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build cart service with item merge, discount application, and inventory reservation integration. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build session store service with token revocation, idle timeout, and concurrent login policy. +0 0 Define verification and readiness surface | Build a secret rotation controller with staged rollout, canary verification, rollback triggers, and explicit blast-radiu +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Plan shared sync core plus Android/Desktop/Web projections with platform-specific | Implement shared sync core plus Android/Desktop/Web projections with platform-specific persistence and transport adapter +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +0 0 Define verification and readiness surface | Build LLM guardrail service for policy checks, PII filtering, and response classification. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a multi-tenant limiter with token bucket + sliding window hybrid policy, burst fairness guarantees, and lock-free +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a Kubernetes manifest linter enforcing probes, limits, and namespace policy. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Define verification and readiness surface | Build an API for file metadata indexing, search by attributes, and soft delete recovery. +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build experiment tracking backend storing params, metrics, artifacts, and compare queries. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build user profile service with avatars, privacy settings, and profile completeness scoring. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +0 0 Define verification and readiness surface | Build batch inference pipeline reading partitions, loading model, and writing scored outputs. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build a JWT auth service with register, login, refresh token rotation, and password hashing. +0 0 Define verification and readiness surface | Build notification preferences API with channel rules, quiet hours, and topic subscriptions. +0 0 Plan Build a CI orchestrator service triggering jobs tracking | Build a CI orchestrator service triggering jobs, tracking states, and emitting summaries. +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build a harness that transpiles class-heavy source modules across multiple target languages and validates class/method p +0 0 Plan telemetry agent for linux arduino targets using shared | Implement telemetry agent for linux + arduino targets using shared serialization core and target-specific runtime wrappe +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build prompt evaluation runner with rubric scoring, aggregate metrics, and reproducible reports. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a feature store service with online/offline reads and point-in-time correctness. +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build wiki backend with page versions, backlinks, and markdown rendering pipeline hooks. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build drift detection worker comparing live feature distributions against reference baselines. +0 0 Define verification and readiness surface | Build form builder backend with schema validation, submissions, and analytics counters. +0 0 Define verification and readiness surface | Build email campaign manager with lists, segmentation, scheduling, and bounce tracking. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +0 0 Define verification and readiness surface | Build an access gateway that evaluates hierarchical allow/deny policies with shadow rules, policy versioning, and explai +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a presence tracker with heartbeat timeout, status transitions, and shard-safe updates. +0 0 Define verification and readiness surface | Build payments API with idempotency keys, ledger-safe transfers, and webhook callbacks. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Plan Build a task runner CLI that reads YAML | Build a task runner CLI that reads YAML tasks and executes dependency-aware command graphs. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build configuration service with versioned configs, staged rollout, and validation checks. +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +0 0 Define verification and readiness surface | Build warehouse picking service with wave assignment, pick confirmation, and shortage handling. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build shipping label worker integrating carrier rates, label generation, and tracking updates. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a metrics aggregation worker with tumbling windows, late data handling, and rollups. +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build backup retention manager with policy tiers, legal hold support, and deletion safety checks. +0 0 Plan Build reconciliation service matching ledger entries to bank | Build reconciliation service matching ledger entries to bank statements with mismatch reporting. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build an ETL pipeline that ingests orders CSV, validates schema, and loads curated tables. +0 0 Define verification and readiness surface | Build search suggest API with prefix index, typo tolerance, and popularity boosting. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build a feature flag API with environments, targeting rules, and rollout percentages. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Plan Replace search ranking pipeline behind feature flag with | Replace search ranking pipeline behind feature flag with staged rollout, telemetry checks, and automatic abort trigger. +0 0 Define verification and readiness surface | Build a training scheduler with queue priorities, resource quotas, and retry policies. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +0 0 Define verification and readiness surface | Build a gateway that validates incoming payloads against versioned contracts, performs semantic constraint checks, and e +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a backup CLI with include/exclude rules, incremental snapshots, and restore command. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +0 0 Define verification and readiness surface | Build pricing engine with currency handling, promotions, and time-bound campaign rules. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +0 0 Define verification and readiness surface | Build an event deduplicator using idempotency keys, time windows, and replay-safe writes. +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +0 0 Define verification and readiness surface | Build dataset version management with lineage metadata, immutable snapshots, and diff operations. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +0 0 Define verification and readiness surface | Build tax report backend aggregating gains/losses and generating filing-year summaries. +0 0 Define verification and readiness surface | Build distributed rate limiter with fixed/sliding window modes and policy APIs. +0 0 Define verification and readiness surface | Build telemetry collector with trace/span ingestion, sampling, and export adapters. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | Build a CRDT-based collaborative notes backend with causal metadata, merge convergence proofs, tombstone compaction, and +0 0 Define verification and readiness surface | Build a loader from object storage to warehouse with schema evolution and retry checkpoints. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +0 0 Define verification and readiness surface | Build access control service with RBAC roles, permissions, and resource-scoped grants. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +3 0 Define verification and readiness surface | Refactor billing pipeline to introduce idempotency keys and rollback checkpoints across ledger service, queue worker, re +0 0 Define verification and readiness surface | Build a CDC replication worker that handles schema evolution, backfill merges, checkpoint consistency, and deterministic +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +0 0 Plan Build drift detection comparing desired and observed infra | Build drift detection comparing desired and observed infra states with actionable diff reports. +1 0 Define verification and readiness surface | Build a ledger reconciliation service with strict double-entry invariants, idempotent replay, rollback checkpoints, and +0 0 Define verification and readiness surface | Build product catalog API with variants, facets, search indexing hooks, and soft delete. +0 0 Define verification and readiness surface | Build a realtime log viewer backend with tail sessions, regex filters, and backpressure. +1 0 Define verification and readiness surface | Build audit log service with immutable events, filtering, export, and retention policies. +1 0 Define verification and readiness surface | Build localization service with key bundles, fallback locales, and missing-key diagnostics. +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a realtime cursor service broadcasting positions with rate limiting and stale cleanup. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +0 0 Define verification and readiness surface | Build a text diff CLI with unified output, word-level mode, and ignore-whitespace option. +0 0 Define verification and readiness surface | Build calendar backend with events, recurrence rules, reminders, and timezone normalization. +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build inventory endpoints with stock reservation, release, and low-stock alerts. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +0 0 Define verification and readiness surface | Build an inference API with model loading, request validation, and batch prediction endpoint. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +0 0 Define verification and readiness surface | Build recommendation API combining collaborative and rule-based fallback candidate generation. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build an order saga orchestrator with compensating actions, timeout escalation, exactly-once message handling, and deter +2 0 Plan Introduce deny-by-default RBAC policy and propagate through backend | Introduce deny-by-default RBAC policy and propagate through backend auth middleware, endpoint guards, frontend route gua +2 0 Plan Build a generator that scaffolds Terraform modules with | Build a generator that scaffolds Terraform modules with inputs, outputs, and policy checks. +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build a .env manager CLI with profile switch, validation, and secure redaction in output. +1 0 Define verification and readiness surface | Build fraud rules engine with rule evaluation graph, score thresholds, and audit trails. +0 0 Define verification and readiness surface | Build a CMS backend with posts, tags, draft/publish workflow, and role-based access control. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +3 0 Define verification and readiness surface | All core features implemented Tests passing +1 0 Define verification and readiness surface | Build review moderation with profanity checks, spam heuristics, and moderation queue endpoints. +0 0 Define verification and readiness surface | Build a log shipping agent with offset persistence, multiline parsing, and retries. +0 0 Define verification and readiness surface | Build a data quality checker with null, range, uniqueness, and referential integrity rules. +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +3 0 Define verification and readiness surface | All core features implemented Tests passing +0 0 Define verification and readiness surface | Build payout scheduler with settlement windows, minimum thresholds, and retry-safe processing. +3 0 Define verification and readiness surface | All core features implemented Tests passing +2 0 Align docs with bootstrap architecture | Build KYC workflow with document checks, status transitions, and reviewer assignment queues. +1 0 Define verification and readiness surface | Build a distributed scheduler with per-resource mutex locks, lease renewal, deadlock avoidance ordering, and determinist +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +1 0 Define verification and readiness surface | Build an incremental build executor over a DAG with content-hash invalidation, topological scheduling, cycle diagnostics +2 0 Align docs with bootstrap architecture | Build document search service with indexing, permissions filtering, and ranking features. +3 0 Define verification and readiness surface | Build a CRDT-based collaborative notes backend with causal metadata, merge convergence proofs, tombstone compaction, and +0 0 Define verification and readiness surface | Build a notes sync API with optimistic concurrency, conflict detection, and change feed endpoints. +0 0 Define verification and readiness surface | Build OCR queue processor with job retries, dead-letter handling, and extraction status API. +0 0 Define verification and readiness surface | Build shared core and projections but omit projection contract fields intentionally. +1 0 Define verification and readiness surface | Build a PII redaction pipeline masking emails, phones, and IDs with audit logging. +3 0 Define verification and readiness surface | All core features implemented Tests passing diff --git a/specialists/data/generated/worker_type_train.tsv b/specialists/data/generated/worker_type_train.tsv new file mode 100644 index 0000000..c91dc0b --- /dev/null +++ b/specialists/data/generated/worker_type_train.tsv @@ -0,0 +1,300 @@ +1 0 `whetstone_get_sync_diagnostics` MCP tool +3 0 Regression fixtures for under-constrained vs capability-limited cases +3 0 Sprint 83 integration summary + regression +1 0 Audit trail and inspection +3 0 Sprint 99 integration summary + regression +3 0 Security Testing Skeleton Generation +2 0 Phase 25b Integration +3 0 Sprint 115 integration summary + regression +0 0 Feedback Loop — Rejection Re-Routing +2 0 Long-horizon roadmap planning engine v2 +2 0 Retry/Escalation Protocol for Constraint Failures +0 0 `BuildErrorRecord` schema +2 0 Phase 26a Integration +3 0 Sprint 179 Integration Summary +0 0 Policy-compliance auto-checkpoint integration +2 0 Refactor grammar generator to use normalized schema IR +3 0 Intake-to-Queue Simulation Harness +3 0 Sprint 50 integration summary + regression +1 0 Human review queue generator from ambiguities +2 0 Scaffold File Generation +3 0 Sprint 85 integration summary + regression +0 0 `whetstone_get_swarm_maintenance_status` MCP tool +3 0 Sprint 157 integration summary + regression +3 0 Full historical regression + post-release checklist +1 0 `whetstone_normalize_diagnostics` runtime implementation +3 0 End-to-end — Python PriorityQueue → C++, Rust, Go +2 0 Phase 12d Integration + Sprint 12 Summary +1 0 False-green diagnostic class +3 0 Sprint 86 integration summary + regression +3 0 Sprint 64 integration summary + regression +3 0 Sprint 142 integration summary + regression +1 0 Priority-focused review queue shaping engine +1 0 Efficiency diagnostics and trend report +0 0 Context slice canonicalization for handoffs +0 0 Context Bundle Format +3 0 Reference implementation harness for IR/evidence specs +1 0 Constraint completeness diagnostics +1 0 Porting review board model +0 0 Maintenance demand estimator +0 0 `whetstone_get_program_kpis` MCP tool +1 0 Tool-surface diagnostics report artifact +3 0 Rust runner adapter for harness +2 0 `whetstone_verify_architecture_consistency` MCP tool +1 0 Security scan orchestrator for generated targets +1 0 Strictness audit utility and baseline report +0 0 Sprint pipeline summary enrichment +0 0 C Parser — Functions, Structs, Enums +1 0 Call Stack and Frame Inspector +3 0 Sprint 74 integration summary + regression +2 0 Phase 25b Integration +0 0 Binary/interface diff engine integration +3 0 Sprint 125 integration summary + regression +1 0 Reviewer quorum policy model for critical approvals +1 0 Conflict preview and impact estimator model +3 0 Test failure classifier +3 0 Benchmark ingestion support for projection schema +0 0 Artifact quality scoring and ranking engine +3 0 Sprint 47 integration summary + regression +0 0 `whetstone_assemble_context` MCP tool +0 0 Legacy modernization dossier generator +0 0 Step Engine +1 0 Agent role contract schema (`analyzer`, `patcher`, `verifier`, `reviewer`) +0 0 `whetstone_trigger_porting_incident_drill` MCP tool +1 0 Lint diagnostics normalization +1 0 AST mutation preview +2 0 Burn-down planning optimizer +1 0 KPI delta analysis engine for block review +2 0 Long-horizon roadmap planning engine v2 +0 0 `whetstone_rollback_constructive_transaction` MCP tool +2 0 Deterministic blocker taxonomy +1 0 Docking Reliability Audit + Deterministic Layout Rules +0 0 `whetstone_run_build_iteration` MCP tool +1 0 Divergence triage model for interop failures +2 0 Phase 36a Integration +0 0 Language capability conformance report artifact +3 0 `whetstone_run_pair_benchmark` MCP tool +1 0 `whetstone_optimize_review_queue` MCP tool +3 0 Sprint 205 Integration Summary +1 0 Loop summary contract diagnostics +2 0 Phase 13a Integration — Layout Persistence +3 0 Sprint 189 Integration Summary +2 0 Continuity drill framework and scoring +2 0 Deterministic blocker taxonomy extension +3 0 Cross-team benchmark comparator +0 0 VB.NET lowering/raising adapters +1 0 Review-Gate Policy Refinement +2 0 Phase 14d Integration + Sprint 14 Summary +0 0 Multi-language compile gate executor +1 0 Dependency and supply-chain audit packet +1 0 Reviewer-focused semantic diff summarizer +1 0 Architect Review Surface for Intake +2 0 Phase 24a Integration +2 0 Long-range blocker taxonomy +2 0 Result Acceptance Protocol +1 0 Patch proposal review report artifact +1 0 Parity diagnostics export +3 0 Debug benchmark harness for fixture suites +1 0 Triage packet bundle model +1 0 Incremental sync diagnostics model +2 0 Swarm maintenance job decomposition model +0 0 Epoch kickoff report artifact +3 0 Sprint 96 integration summary + regression +2 0 Phase 32a Integration +2 0 Self-Hosting Progress Report + Phase Integration +2 0 Phase 13a Integration — Layout Persistence +0 0 Sprint 28 Integration + Summary +2 0 Migration Plan Generator +2 0 Phase 20a Integration +3 0 End-to-end replay + rollback scenario suite +3 0 C++ runner adapter for harness +1 0 Safety Audit via Annotations +2 0 Phase 33a Integration +3 0 Sprint 79 integration summary + regression +0 0 Lisp lowering/raising adapters +0 0 `whetstone_run_porting_gates` MCP tool +1 0 Explainability compression model for rapid review +2 0 Phase 32a Integration +3 0 Sprint 51 integration summary + regression +3 0 Sprint 175 Integration Summary +3 0 Sprint 146 integration summary + regression +2 0 Deterministic blocker taxonomy extension +3 0 Sprint 139 integration summary + regression +3 0 Sprint 90 integration summary + full-program regression +3 0 Benchmark publication bundle +2 0 Transpilation failure taxonomy schema +3 0 Sprint 138 integration summary + regression +1 0 Architect Review Interface +1 0 Real adapter diagnostics in status tool +0 0 Kotlin Parser +1 0 Remediation router from gate diagnostics to tool actions +0 0 Family promotion checks (experimental->beta) +2 0 Phase 16a Integration +3 0 Sprint 73 integration summary + regression +2 0 Phase 13d Integration + Sprint 13 Summary +0 0 AnnotationValidator — Subject 5-8 Rules +0 0 Deterministic replay pack composer +1 0 Assurance review dossier template +3 0 Auto-tuning proposal generator from benchmark deltas +1 0 `whetstone_get_distributed_triage_queue` MCP tool +1 0 KPI anomaly triage model +2 0 Phase 36a Integration +2 0 Module Decomposition Engine +1 0 Scope ambiguity diagnostics +2 0 Phase 38a Integration +1 0 Governance and audit report bundle +2 0 Phase 13b Integration — Visual Consistency +3 0 CI integration in build/test flow +0 0 Zero-trust policy model for execution surfaces +1 0 Patch execution audit bundle +2 0 Phase 30a Integration +1 0 Drift audit report artifact +0 0 `whetstone_marketplace_search` MCP tool +3 0 Cross-language class emission parity tests +3 0 Test output parser — gtest format +0 0 `whetstone_get_debug_campaign_status` MCP tool +1 0 Dynamic family acceptance report + review queue wiring +3 0 Sprint 191 Integration Summary +2 0 Generator Updates + Phase 11c Integration +1 0 Review effort estimator from ambiguity/risk packets +3 0 Sprint 152 integration summary + regression +2 0 Candidate generation engine from failure taxonomy +0 0 Ambiguity packet model (`assumed`, `unknown`, `conflict`) +3 0 Sprint 198 Integration Summary +1 0 Memory Inspector UI Model +0 0 Argument/result canonicalizer model +3 0 Sprint 89 integration summary + regression +1 0 Divergence classification and triage model +0 0 `whetstone_run_constructive_loop` MCP tool +1 0 Dependency Audit Annotations +3 0 Benchmark runner and metrics export +3 0 Sprint 62 integration summary + regression +1 0 Review Comparison View +1 0 Review SLA policy integration +0 0 Failure injection and safety validation model +1 0 Intent drift diagnostics export +3 0 Sprint 186 Integration Summary +0 0 Auto Type Deduction +2 0 Scope and Milestone Decomposer +2 0 Phase 25a Integration +1 0 Constraint Violation Diagnostics +2 0 Refactor pattern schema and metadata model +1 0 `whetstone_normalize_diagnostics` MCP tool +0 0 Replay/transaction reliability report artifact +1 0 Reviewer effectiveness report artifact +1 0 Post-incident continuity audit report model +1 0 Entropy scanner — editor-side +2 0 Phase 13c Integration — Keyboard Shortcuts Help Panel +3 0 Test gate runner and minimal harness injection +2 0 Backend framework adapter hooks +0 0 Enhanced evidence requirements engine +1 0 `whetstone_preview_regenerated_diff` MCP tool +1 0 Hint safety guardrails and audit fields +2 0 Refactor intent annotation model +3 0 Sprint 137 integration summary + regression +1 0 Reporting templates for C++ review teams +0 0 Generic/monomorphization intent model +0 0 `whetstone_compare_mcp_replays` MCP tool +0 0 Memory Model Translation +0 0 Schema evolution compatibility checker +0 0 Closeout consistency gate +0 0 Real-time constraint model (`deadline`, `jitter`, `period`) +2 0 Workflow Session Protocol +0 0 Include/import ordering normalization model +2 0 Deprecation/succession lifecycle framework +0 0 Trace Timeline Model +1 0 Decision consistency analyzer across reviewers +1 0 Profile compliance auditor integration +0 0 `whetstone_plan_rollout_stage` MCP tool +0 0 Top-level strictness gate +0 0 Strict queue gate for execution contracts +3 0 Sprint 58 integration summary + regression +3 0 Sprint 48 integration summary + regression +0 0 `whetstone_detect_text_ast_conflicts_generic` MCP tool +2 0 Phase 19a Integration +1 0 `whetstone_get_review_load_status` MCP tool +3 0 Phase 19b Integration + Sprint Summary +3 0 Sprint 200 Integration Summary +0 0 Async/await state intent lowering +1 0 Failure triage score model +0 0 Plan fingerprinting and replay checks +1 0 MCP replay diagnostics artifact +0 0 Plugin compatibility resolver (core/API/version) +1 0 Diagnostic normalization canonical model +3 0 Test-to-source locator +0 0 Debug action budget model +2 0 Scaffold File Generation +1 0 Canonical diagnostic normalization from provider/LSP output +2 0 Backend framework adapter hooks +3 0 Sprint 162 Integration Summary +2 0 Phase 39a Integration +0 0 ADT + pattern matching canonical lowering +2 0 Verification gate integration for architecture artifacts +0 0 Modernization RPC + MCP +2 0 Phase 19a Integration +3 0 Convention Extractor + Validator (tests: unit, negative, regression) +1 0 `whetstone_preview_text_ast_merge` MCP tool +0 0 ABI gate bindings to tier promotions +2 0 Architecture claim linkage engine +0 0 Permission conformance report artifact +2 0 Phase 20a Integration +2 0 Governance model refresh framework +1 0 Sync diagnostics normalization model +3 0 Sprint 63 integration summary + regression +0 0 `whetstone_suggest_build_fix` MCP tool +3 0 A/B + benchmark validation artifact update +0 0 `whetstone_plan_tool_migrations` MCP tool +3 0 Full regression + architecture gate +0 0 C interop adapter deepening +3 0 Phase 10e Integration Tests +2 0 Phase 29a Integration +2 0 Scope and Milestone Decomposer +0 0 Multi-check gate orchestrator model +1 0 Lint executor integration +3 0 Sprint 118 integration summary + regression +1 0 Dropped-content diagnostics packet +0 0 Execution trace and recovery packet export +2 0 Phase 18a Integration +3 0 Sprint 45 Integration Summary +2 0 Deterministic blocker taxonomy +2 0 Refactor pattern schema and metadata model +1 0 Reviewer decision ledger integration +2 0 Phase 28a Integration +2 0 Phase 37a Integration +3 0 Sprint 114 integration summary + regression +2 0 `whetstone_publish_roadmap_epoch` MCP tool +1 0 `whetstone_review_porting_decision` MCP tool +2 0 Language-specific method body scaffolding for core patterns +1 0 Audit trail completeness checker +1 0 C++ include fixer from gate diagnostics +3 0 End-to-end capability inventory and gap scan +1 0 Static registry scanner for `register*Tools` blocks +2 0 Phase 31a Integration +1 0 `whetstone_rank_failure_triage_actions` MCP tool +3 0 Timing regression runner integration +2 0 Phase 40a Integration +1 0 Reviewer load and fatigue model +2 0 Swarm maintenance job decomposition model +2 0 Target projection blocker taxonomy +0 0 Annotate Whetstone's Own Code +2 0 Phase 15c Integration + Sprint 15 Summary +0 0 Epoch block health re-baseline model +3 0 Sprint 159 integration summary + regression +1 0 Human Review Interface via MCP +0 0 Transpile Whetstone Modules +0 0 Capability matrix model +2 0 Phase 14d Integration + Sprint 14 Summary +3 0 Prior-Step Context Injector (tests: unit, integration) +0 0 Unsafe-practice detection and quarantine +3 0 Sprint 127 integration summary + regression +1 0 `TaskitemQualityAuditor` — batch report +3 0 Systemic regression detector +0 0 Debug campaign spec model +3 0 `whetstone_get_execution_attestation` MCP tool +3 0 Sprint 72 integration summary + regression +2 0 Self-Hosting Progress Report + Phase Integration +0 0 C# Parser +1 0 Ambiguity triage UI model and API +0 0 F# Parser +0 0 Result-to-ObservationRecord mapper +0 0 Self-service escalation handoff hooks diff --git a/specialists/docs/ARCHITECTURE.md b/specialists/docs/ARCHITECTURE.md new file mode 100644 index 0000000..b94821c --- /dev/null +++ b/specialists/docs/ARCHITECTURE.md @@ -0,0 +1,197 @@ +# Whetstone Specialist Fleet — Architecture + +**Date:** 2026-03-28 +**Status:** Phase 1 — first specialist training + +--- + +## The Core Idea + +The current Whetstone pipeline makes every bounded decision with one large LLM call. +Each call is ~seconds, sequential, and expensive. Many of these decisions are +bounded-output classifications — the exact problem that WhetstoneAI Fabricate's +~213K-parameter (~800KB) specialists solve in microseconds. + +**Hypothesis:** Replace bounded-output LLM decisions in the Whetstone taskitem +pipeline with a fleet of micro-specialists, each ~800KB, running in parallel. + +### Why It Works + +The WhetstoneAI RSA experiment demonstrated: +- 96.3% in-distribution accuracy on domain classification +- 76.7% OOD accuracy after replay hardening (up from 44.8%) +- Inference time: microseconds per query on GPU, 1–5ms on CPU (numpy) +- Model size: ~213K params, ~800KB per checkpoint +- RTX 3060 (12GB VRAM): can hold ~14,000 specialists resident simultaneously + +The Whetstone taskitem pipeline already has: +- Bounded enum decision points (verification type, prerequisite ops, worker type...) +- Recorded task completions (18,519 pipeline runs) +- Concurrency/dependency architecture (lock-based pool with parallel workers) + +This means the specialists can map 1:1 onto the pipeline's decision points and +run in parallel with the existing task dependency graph. + +--- + +## Decision Points Amenable to Specialists + +### Tier 1: Text-Input Classifiers (primary target) + +These decisions receive a task description and output a bounded choice. +They currently run inside the LLM and are the highest-value targets. + +| Specialist | Input | Output | Classes | +|-----------|-------|--------|---------| +| `verification_type` | task step description | verification type | unit / integration / schema / smoke / docs | +| `prereq_op_selector` | task description | prerequisite op set | validate-intake / resolve-dependencies / architect-review / manual-approval | +| `worker_type` | task description | worker role | implementer / reviewer / architect / qa | +| `gap_class` | taskitem issues list | gap classification | under_constrained / env_gap / planning_gap / cross_project / generator_cap | + +### Tier 2: Structural Specialists (formula approximators) + +These learn the deterministic scoring formulas — useful for ultra-fast pre-screening +without spinning up the full validator. + +| Specialist | Input | Output | Notes | +|-----------|-------|--------|-------| +| `confidence_tier` | (conflict_count, ambiguity_count, deps, prereqs) | high / medium / low | Learns the deduction formula | +| `escalation_predictor` | (confidence, ambiguity_count) | escalate: bool | Binary: confidence < 60 OR ambiguity > 0 | +| `quality_score_tier` | task features | pass / warn / fail | Learns SelfContainmentScorer cutoffs | + +### Tier 3: Future Specialists (need data collection) + +| Specialist | Input | Output | Blocker | +|-----------|-------|--------|---------| +| `library_selector` | task intent + available libs | selected library | No labeled data yet | +| `milestone_count` | requirement count + conflicts | 1 / 2 / 3 / 4+ milestones | Need more diverse pipeline runs | +| `task_position_to_dependency` | task index + total count | has_dep: bool | Imbalanced data (97% no-dep) | + +--- + +## Architecture: Fleet Model + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Whetstone MCP Request │ +│ (generate_taskitems, validate_taskitem, ...) │ +└────────────────────────┬────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ Specialist Dispatcher (Python: infer.py) │ +│ │ +│ Task text ──► verification_type_specialist ──► unit │ +│ Task text ──► prereq_op_specialist ──► [validate, ...] │ +│ Task text ──► worker_type_specialist ──► implementer │ +│ Features ──► confidence_tier_specialist ──► high │ +│ │ +│ All specialists run in parallel via thread pool │ +└──────────────┬──────────────────────────────────────────────────┘ + │ aggregated results + ▼ +┌─────────────────────────────────────────────────────────────────┐ +│ Whetstone C++ Runtime │ +│ (enriches taskitem fields with specialist predictions) │ +└─────────────────────────────────────────────────────────────────┘ +``` + +### Specialist Model Spec (all identical architecture) + +``` +Architecture: Transformer (D=128, L=1, heads=1, seq_len=96) +Parameters: ~213K +Size: ~832KB per checkpoint +Inference: <1ms CPU (numpy), ~50µs GPU +Training: 1K–5K steps, grokking detection, weight decay +``` + +Reuses the WhetstoneAI Fabricate binary (`whetstone_fabricate`) and training harness +(`run_grokking_until.py`) without modification. + +--- + +## Training Pipeline + +``` +sprint_plans/*.md + │ + ▼ +scripts/gen_verification_type_data.py ← TSV extractor (one per specialist) + │ + ▼ +data/generated/verification_type_train.tsv (label0text) +data/generated/verification_type_eval.tsv + │ + ▼ +scripts/train_verification_type.sh ← wraps run_grokking_until.py + │ + ▼ +/mnt/storage/fabricate_runs/whetstone_verification_type/ + checkpoint.bin (weights + optional Adam state) + fabricate.db (vocabulary) + │ + ▼ +specialists/scripts/infer.py ← Python inference wrapper + │ + ▼ +Whetstone MCP tool integration (future) +``` + +--- + +## Data Sources + +### Current (18,519 pipeline run records) +`training_data/lora/taskitem_pipeline_runs.jsonl` + +**Limitation:** Almost entirely homogeneous (2 task types repeated, no text diversity). +Useful for structural/formula specialists (Tier 2). NOT useful for text-based classifiers. + +### Primary for Text Specialists +`sprint*_plan.md` files (260 files, ~2,600 labeled step descriptions) + +Each step description has an implicit verification type based on its content: +- `### Step NNN: whetstone_foo MCP tool` → integration +- `### Step NNN: Schema for X` → schema +- `### Step NNN: Sprint N integration summary + regression` → smoke +- `### Step NNN: X documentation packet` → docs +- Everything else → unit + +### Future Data Collection +Add richer recording to `run_sprint_taskitem_pipeline.sh`: +- Capture per-step verification type decisions +- Record library selection choices +- Record milestone count decisions + +--- + +## Current Status + +| Specialist | Data Source | Training Status | Accuracy | +|-----------|------------|-----------------|---------| +| `verification_type` | sprint plans (978 train, 172 eval) | **TRAINING** | TBD | +| All others | — | planned | — | + +--- + +## Integration Plan (Future) + +1. **Phase 1 (now):** Train verification_type specialist, validate accuracy +2. **Phase 2:** Wire into `whetstone_validate_taskitem` as a pre-prediction step +3. **Phase 3:** Train prereq_op and confidence_tier specialists +4. **Phase 4:** Build Python dispatcher that runs all specialists in parallel +5. **Phase 5:** Expose as new MCP tool `whetstone_specialist_predict` +6. **Phase 6:** Replace LLM calls for bounded decisions with specialist fleet + +--- + +## Connection to WhetstoneAI Fabricate + +This project reuses the Fabricate infrastructure directly: +- Binary: `/home/bill/Documents/WhetstoneAI_Fabricate/build/whetstone_fabricate` +- Harness: `/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py` +- Eval tools: `/home/bill/Documents/WhetstoneAI_Fabricate/tools/eval_multiclass.py` +- Inference: `/home/bill/Documents/WhetstoneAI_Fabricate/tools/eval_with_vocab.py` + +No code changes to Fabricate are needed. It is consumed as a training and inference substrate. diff --git a/specialists/eval/eval_specialist.py b/specialists/eval/eval_specialist.py new file mode 100644 index 0000000..82978eb --- /dev/null +++ b/specialists/eval/eval_specialist.py @@ -0,0 +1,282 @@ +#!/usr/bin/env python3 +""" +eval_specialist.py + +Extended evaluation for whetstone specialist checkpoints. +Produces confusion matrix, per-class accuracy, confidence distribution, +and threshold analysis (accepted precision + abstain rate). + +Output: JSON to stdout or --out file. + +Usage: + python3 specialists/eval/eval_specialist.py \ + --checkpoint /mnt/storage/fabricate_runs/whetstone_worker_type/checkpoint.bin \ + --db /mnt/storage/fabricate_runs/whetstone_worker_type/fabricate.db \ + --dataset specialists/data/generated/worker_type_eval.tsv \ + --labels "implementer,reviewer,architect,qa" \ + [--out specialists/eval/results/worker_type_eval.json] + [--thresholds 0.5,0.7,0.8,0.9] + [--seq-len 96] +""" + +import argparse +import json +import sys +import time +from collections import defaultdict +from pathlib import Path + +import numpy as np + +FABRICATE_TOOLS = Path("/home/bill/Documents/WhetstoneAI_Fabricate/tools") +sys.path.insert(0, str(FABRICATE_TOOLS)) + +from eval_with_vocab import ( + load_checkpoint, load_vocab, load_layers, build_runtime, + tokenize_batch, forward_batch, +) + + +def softmax(x): + e = np.exp(x - x.max(axis=-1, keepdims=True)) + return e / e.sum(axis=-1, keepdims=True) + + +def load_labels(s): + p = Path(s) + if p.exists(): + return [t.strip() for t in p.read_text().strip().split(",") if t.strip()] + return [t.strip() for t in s.split(",") if t.strip()] + + +def evaluate(checkpoint_path, db_path, dataset_path, label_tokens, + seq_len=96, batch_size=256): + weights, _ = load_checkpoint(checkpoint_path) + _, str_to_id = load_vocab(db_path) + layers = load_layers(db_path) + + vocab_size = len(str_to_id) + _, emb_cnt = layers["token_embedding"] + D = emb_cnt // vocab_size + _, ff1_cnt = layers["block0.ff1_weight"] + F = ff1_cnt // D + L = sum(1 for k in layers if k.startswith("block") and k.endswith(".qkv_weight")) + cfg = dict(seq_len=seq_len, D=D, F=F, V=vocab_size, L=L) + runtime = build_runtime(weights, layers, cfg) + + label_vocab_ids = [] + missing = [] + for tok in label_tokens: + vid = str_to_id.get(tok) + if vid is None: + missing.append(tok) + vid = 1 + label_vocab_ids.append(vid) + label_vocab_ids_arr = np.array(label_vocab_ids, dtype=np.int32) + + labels_gt, texts = [], [] + with open(dataset_path) as f: + for line in f: + parts = line.strip().split("\t", 2) + if len(parts) < 3: + continue + labels_gt.append(int(parts[0])) + texts.append(parts[2]) + + labels_gt = np.array(labels_gt, dtype=np.int32) + n = len(labels_gt) + n_labels = len(label_tokens) + + all_preds = [] + all_confs = [] # max probability for predicted class + + t0 = time.perf_counter() + for start in range(0, n, batch_size): + stop = min(start + batch_size, n) + token_ids = tokenize_batch(texts[start:stop], str_to_id, seq_len) + logits = forward_batch(token_ids, weights, layers, cfg, runtime=runtime) + label_logits = logits[:, label_vocab_ids_arr] + probs = softmax(label_logits.astype(np.float64)) + pred_idx = np.argmax(probs, axis=1) + conf = probs[np.arange(len(pred_idx)), pred_idx] + all_preds.append(pred_idx) + all_confs.append(conf) + + eval_latency_ms = (time.perf_counter() - t0) * 1000 / max(n, 1) + + preds = np.concatenate(all_preds) + confs = np.concatenate(all_confs) + correct = (preds == labels_gt) + + # --- Overall accuracy --- + overall_acc = float(correct.mean() * 100) + + # --- Per-class accuracy --- + per_class = {} + for i, name in enumerate(label_tokens): + mask = labels_gt == i + if mask.sum() == 0: + per_class[name] = {"acc": None, "n": 0} + else: + per_class[name] = { + "acc": float(correct[mask].mean() * 100), + "n": int(mask.sum()), + } + + # --- Confusion matrix --- + confusion = np.zeros((n_labels, n_labels), dtype=int) + for gt, pred in zip(labels_gt, preds): + confusion[gt, pred] += 1 + + # Adjacent confusion (common mistake pairs) + confusion_summary = [] + for i in range(n_labels): + for j in range(n_labels): + if i != j and confusion[i, j] > 0: + confusion_summary.append({ + "gt": label_tokens[i], + "pred": label_tokens[j], + "count": int(confusion[i, j]), + "gt_total": int((labels_gt == i).sum()), + }) + confusion_summary.sort(key=lambda x: -x["count"]) + + # --- Confidence distribution --- + conf_bins = [0.0, 0.5, 0.6, 0.7, 0.8, 0.9, 0.95, 1.0] + conf_hist = {} + for lo, hi in zip(conf_bins[:-1], conf_bins[1:]): + mask = (confs >= lo) & (confs < hi) + n_in_bin = int(mask.sum()) + if n_in_bin == 0: + continue + acc_in_bin = float(correct[mask].mean() * 100) if n_in_bin > 0 else None + conf_hist[f"{lo:.2f}-{hi:.2f}"] = {"n": n_in_bin, "acc": acc_in_bin} + + # --- Threshold analysis --- + threshold_results = [] + for threshold in [0.5, 0.6, 0.7, 0.8, 0.9]: + accepted_mask = confs >= threshold + n_accepted = int(accepted_mask.sum()) + n_abstained = n - n_accepted + if n_accepted == 0: + threshold_results.append({ + "threshold": threshold, + "accepted": 0, + "abstain_rate": 1.0, + "accepted_accuracy": None, + }) + else: + acc = float(correct[accepted_mask].mean() * 100) + threshold_results.append({ + "threshold": threshold, + "accepted": n_accepted, + "abstain_rate": round(n_abstained / n, 3), + "accepted_accuracy": round(acc, 1), + }) + + # --- Guardrail assessment --- + # Find lowest threshold where accepted_accuracy >= 95% + guardrail_threshold = None + guardrail_abstain_rate = None + for tr in threshold_results: + if tr["accepted_accuracy"] is not None and tr["accepted_accuracy"] >= 95.0: + guardrail_threshold = tr["threshold"] + guardrail_abstain_rate = tr["abstain_rate"] + break + + # --- Deployment verdict --- + if overall_acc >= 97: + verdict = "deploy_direct" + elif guardrail_threshold is not None and guardrail_abstain_rate is not None and guardrail_abstain_rate <= 0.30: + verdict = "deploy_with_guardrails" + elif overall_acc >= 80: + verdict = "scale_more" + elif len(confusion_summary) > 0 and confusion_summary[0]["count"] / n > 0.10: + verdict = "decompose_or_relabel" + else: + verdict = "needs_investigation" + + return { + "overall_accuracy": round(overall_acc, 1), + "n_eval": n, + "per_class": per_class, + "confusion_summary": confusion_summary[:10], + "confidence_histogram": conf_hist, + "threshold_analysis": threshold_results, + "guardrail_threshold": guardrail_threshold, + "guardrail_abstain_rate": guardrail_abstain_rate, + "eval_latency_ms_per_example": round(eval_latency_ms, 3), + "missing_label_tokens": missing, + "model_config": {"D": D, "F": F, "L": L, "vocab": vocab_size}, + "verdict": verdict, + } + + +def print_report(result, labels): + print(f"\n{'='*60}") + print(f" Overall accuracy: {result['overall_accuracy']:.1f}% (n={result['n_eval']})") + print(f" Verdict: {result['verdict']}") + print(f"{'='*60}") + + print("\nPer-class accuracy:") + for name, stat in result["per_class"].items(): + if stat["acc"] is None: + print(f" {name:20s} (no examples)") + else: + print(f" {name:20s} {stat['acc']:5.1f}% (n={stat['n']})") + + if result["confusion_summary"]: + print("\nTop confusions (gt → pred):") + for c in result["confusion_summary"][:6]: + pct = c["count"] / c["gt_total"] * 100 + print(f" {c['gt']:20s} → {c['pred']:20s} {c['count']}x ({pct:.0f}% of gt class)") + + print("\nThreshold analysis:") + print(f" {'thresh':>6} {'accepted':>8} {'abstain%':>8} {'acc@thresh':>10}") + for tr in result["threshold_analysis"]: + abstain_pct = tr["abstain_rate"] * 100 + acc = f"{tr['accepted_accuracy']:.1f}%" if tr["accepted_accuracy"] is not None else " n/a" + print(f" {tr['threshold']:>6.2f} {tr['accepted']:>8} {abstain_pct:>7.1f}% {acc:>10}") + + if result["guardrail_threshold"]: + print(f"\n ✓ Guarded deployment viable at threshold={result['guardrail_threshold']:.2f}" + f" (abstain {result['guardrail_abstain_rate']*100:.0f}%)") + else: + print("\n ✗ No threshold achieves 95% accepted accuracy") + + print(f"\nEval latency: {result['eval_latency_ms_per_example']:.3f} ms/example") + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--checkpoint", required=True) + ap.add_argument("--db", required=True) + ap.add_argument("--dataset", required=True) + ap.add_argument("--labels", required=True) + ap.add_argument("--seq-len", type=int, default=96) + ap.add_argument("--batch-size", type=int, default=256) + ap.add_argument("--out", default=None, help="Write JSON result to file") + ap.add_argument("--quiet", action="store_true", help="Suppress human-readable output") + args = ap.parse_args() + + label_tokens = load_labels(args.labels) + + result = evaluate( + args.checkpoint, args.db, args.dataset, + label_tokens, seq_len=args.seq_len, batch_size=args.batch_size, + ) + + if not args.quiet: + print_report(result, label_tokens) + + if args.out: + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + out_path.write_text(json.dumps(result, indent=2)) + print(f"\nJSON → {args.out}") + + print(json.dumps(result, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/specialists/eval/eval_specialist_pt.py b/specialists/eval/eval_specialist_pt.py new file mode 100644 index 0000000..7a0dba2 --- /dev/null +++ b/specialists/eval/eval_specialist_pt.py @@ -0,0 +1,296 @@ +#!/usr/bin/env python3 +""" +eval_specialist_pt.py + +Extended evaluation for PyTorch specialist checkpoints. +Drop-in replacement for eval_specialist.py, but loads checkpoint.pt instead +of checkpoint.bin + fabricate.db. + +Output: JSON to stdout (and optionally --out file). + +Usage: + python3 specialists/eval/eval_specialist_pt.py \ + --checkpoint /tmp/runs/worker_type/checkpoint.pt \ + --dataset specialists/data/generated/worker_type_eval.tsv \ + --labels "implementer,reviewer,architect,qa" \ + [--out specialists/eval/results/worker_type_eval.json] \ + [--quiet] +""" + +import argparse +import json +import time +from pathlib import Path + +import numpy as np +import torch +import torch.nn as nn +from torch.utils.data import DataLoader, Dataset + + +# ── Copied from train_specialist_pt.py ─────────────────────────────────────── + +def _tokenize(text): + return text.lower().split() + + +class Vocab: + PAD, UNK = 0, 1 + + def __init__(self, str_to_id=None): + if str_to_id is not None: + self.str_to_id = str_to_id + self.id_to_str = [None] * len(str_to_id) + for s, i in str_to_id.items(): + self.id_to_str[i] = s + else: + self.str_to_id = {"": 0, "": 1} + self.id_to_str = ["", ""] + + def encode(self, text, seq_len): + ids = [self.str_to_id.get(t, self.UNK) for t in _tokenize(text)] + ids = ids[:seq_len] + ids += [self.PAD] * (seq_len - len(ids)) + return ids + + def __len__(self): + return len(self.id_to_str) + + +class SpecialistModel(nn.Module): + def __init__(self, vocab_size, hidden_dim, layers, heads, n_labels, seq_len): + super().__init__() + self.seq_len = seq_len + self.tok_emb = nn.Embedding(vocab_size, hidden_dim, padding_idx=0) + self.pos_emb = nn.Embedding(seq_len, hidden_dim) + encoder_layer = nn.TransformerEncoderLayer( + d_model=hidden_dim, + nhead=heads, + dim_feedforward=hidden_dim * 4, + dropout=0.0, + batch_first=True, + ) + self.encoder = nn.TransformerEncoder(encoder_layer, num_layers=layers) + self.head = nn.Linear(hidden_dim, n_labels) + + def forward(self, x): + pad_mask = (x == 0) + pos = torch.arange(x.size(1), device=x.device).unsqueeze(0) + h = self.tok_emb(x) + self.pos_emb(pos) + h = self.encoder(h, src_key_padding_mask=pad_mask) + lengths = (~pad_mask).float().sum(dim=1, keepdim=True).clamp(min=1) + h = (h * (~pad_mask).unsqueeze(-1).float()).sum(dim=1) / lengths + return self.head(h) + + +# ── Eval ────────────────────────────────────────────────────────────────────── + +def softmax(x): + e = np.exp(x - x.max(axis=-1, keepdims=True)) + return e / e.sum(axis=-1, keepdims=True) + + +def evaluate(checkpoint_path, dataset_path, label_tokens, seq_len=96, batch_size=256): + device = torch.device("cuda" if torch.cuda.is_available() else "cpu") + + ckpt = torch.load(checkpoint_path, map_location=device) + cfg = ckpt["config"] + + # Reconstruct vocab from checkpoint + vocab_obj = ckpt.get("vocab_obj") + if vocab_obj is None: + vocab_obj = Vocab(str_to_id=ckpt["vocab"]) + + model = SpecialistModel( + vocab_size=cfg["vocab_size"], + hidden_dim=cfg["hidden_dim"], + layers=cfg["layers"], + heads=cfg["heads"], + n_labels=cfg["n_labels"], + seq_len=cfg["seq_len"], + ).to(device) + model.load_state_dict(ckpt["model"]) + model.eval() + + # Load eval data + labels_gt, texts = [], [] + with open(dataset_path) as f: + for line in f: + parts = line.strip().split("\t", 2) + if len(parts) < 3: + continue + labels_gt.append(int(parts[0])) + texts.append(parts[2]) + + n = len(labels_gt) + labels_gt_arr = np.array(labels_gt, dtype=np.int32) + + # Run inference + all_preds = [] + all_confs = [] + t0 = time.perf_counter() + + with torch.no_grad(): + for start in range(0, n, batch_size): + batch_texts = texts[start:start + batch_size] + ids = [vocab_obj.encode(t, cfg["seq_len"]) for t in batch_texts] + x = torch.tensor(ids, dtype=torch.long, device=device) + logits = model(x).cpu().numpy().astype(np.float64) + probs = softmax(logits) + pred = np.argmax(probs, axis=1) + conf = probs[np.arange(len(pred)), pred] + all_preds.append(pred) + all_confs.append(conf) + + eval_latency_ms = (time.perf_counter() - t0) * 1000 / max(n, 1) + preds = np.concatenate(all_preds) + confs = np.concatenate(all_confs) + correct = (preds == labels_gt_arr) + overall_acc = float(correct.mean() * 100) + + # Per-class accuracy + per_class = {} + for i, name in enumerate(label_tokens): + mask = labels_gt_arr == i + if mask.sum() == 0: + per_class[name] = {"acc": None, "n": 0} + else: + per_class[name] = { + "acc": float(correct[mask].mean() * 100), + "n": int(mask.sum()), + } + + # Confusion matrix + n_labels = len(label_tokens) + confusion = np.zeros((n_labels, n_labels), dtype=int) + for gt, pred in zip(labels_gt_arr, preds): + confusion[gt, pred] += 1 + + confusion_summary = [] + for i in range(n_labels): + for j in range(n_labels): + if i != j and confusion[i, j] > 0: + confusion_summary.append({ + "gt": label_tokens[i], + "pred": label_tokens[j], + "count": int(confusion[i, j]), + "gt_total": int((labels_gt_arr == i).sum()), + }) + confusion_summary.sort(key=lambda x: -x["count"]) + + # Threshold analysis + threshold_results = [] + for threshold in [0.5, 0.6, 0.7, 0.8, 0.9]: + accepted_mask = confs >= threshold + n_accepted = int(accepted_mask.sum()) + n_abstained = n - n_accepted + if n_accepted == 0: + threshold_results.append({ + "threshold": threshold, "accepted": 0, + "abstain_rate": 1.0, "accepted_accuracy": None, + }) + else: + acc = float(correct[accepted_mask].mean() * 100) + threshold_results.append({ + "threshold": threshold, + "accepted": n_accepted, + "abstain_rate": round(n_abstained / n, 3), + "accepted_accuracy": round(acc, 1), + }) + + # Guardrail assessment (lowest threshold with ≥95% accepted accuracy) + guardrail_threshold = None + guardrail_abstain_rate = None + for tr in threshold_results: + if tr["accepted_accuracy"] is not None and tr["accepted_accuracy"] >= 95.0: + guardrail_threshold = tr["threshold"] + guardrail_abstain_rate = tr["abstain_rate"] + break + + # Deployment verdict + if overall_acc >= 97: + verdict = "deploy_direct" + elif guardrail_threshold is not None and guardrail_abstain_rate is not None and guardrail_abstain_rate <= 0.30: + verdict = "deploy_with_guardrails" + elif overall_acc >= 80: + verdict = "scale_more" + elif confusion_summary and confusion_summary[0]["count"] / n > 0.10: + verdict = "decompose_or_relabel" + else: + verdict = "needs_investigation" + + return { + "overall_accuracy": round(overall_acc, 1), + "n_eval": n, + "per_class": per_class, + "confusion_summary": confusion_summary[:10], + "threshold_analysis": threshold_results, + "guardrail_threshold": guardrail_threshold, + "guardrail_abstain_rate": guardrail_abstain_rate, + "eval_latency_ms_per_example": round(eval_latency_ms, 3), + "model_config": cfg, + "verdict": verdict, + } + + +def print_report(result, labels): + print(f"\n{'='*60}") + print(f" Overall accuracy: {result['overall_accuracy']:.1f}% (n={result['n_eval']})") + print(f" Verdict: {result['verdict']}") + print(f"{'='*60}") + print("\nPer-class accuracy:") + for name, stat in result["per_class"].items(): + if stat["acc"] is None: + print(f" {name:20s} (no examples)") + else: + print(f" {name:20s} {stat['acc']:5.1f}% (n={stat['n']})") + if result["confusion_summary"]: + print("\nTop confusions (gt → pred):") + for c in result["confusion_summary"][:6]: + pct = c["count"] / c["gt_total"] * 100 + print(f" {c['gt']:20s} → {c['pred']:20s} {c['count']}x ({pct:.0f}% of gt class)") + print("\nThreshold analysis:") + print(f" {'thresh':>6} {'accepted':>8} {'abstain%':>8} {'acc@thresh':>10}") + for tr in result["threshold_analysis"]: + abstain_pct = tr["abstain_rate"] * 100 + acc = f"{tr['accepted_accuracy']:.1f}%" if tr["accepted_accuracy"] is not None else " n/a" + print(f" {tr['threshold']:>6.2f} {tr['accepted']:>8} {abstain_pct:>7.1f}% {acc:>10}") + if result["guardrail_threshold"]: + print(f"\n Guarded deployment viable at threshold={result['guardrail_threshold']:.2f}" + f" (abstain {result['guardrail_abstain_rate']*100:.0f}%)") + else: + print("\n No threshold achieves 95% accepted accuracy") + print(f"\nEval latency: {result['eval_latency_ms_per_example']:.3f} ms/example") + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--checkpoint", required=True) + ap.add_argument("--dataset", required=True) + ap.add_argument("--labels", required=True) + ap.add_argument("--seq-len", type=int, default=96) + ap.add_argument("--batch-size", type=int, default=256) + ap.add_argument("--out", default=None) + ap.add_argument("--quiet", action="store_true") + # Accept --db for compatibility with old callers (ignored) + ap.add_argument("--db", default=None) + args = ap.parse_args() + + label_tokens = [l.strip() for l in args.labels.split(",")] + result = evaluate(args.checkpoint, args.dataset, label_tokens, + seq_len=args.seq_len, batch_size=args.batch_size) + + if not args.quiet: + print_report(result, label_tokens) + + if args.out: + out_path = Path(args.out) + out_path.parent.mkdir(parents=True, exist_ok=True) + out_path.write_text(json.dumps(result, indent=2)) + print(f"\nJSON → {args.out}") + + print(json.dumps(result, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/specialists/eval/results/verification_type_sweep.log b/specialists/eval/results/verification_type_sweep.log new file mode 100644 index 0000000..0ba85e6 --- /dev/null +++ b/specialists/eval/results/verification_type_sweep.log @@ -0,0 +1,10 @@ + +============================================================ + Tier: small_plus hidden=256 layers=2 + Run dir: /mnt/storage/fabricate_runs/whetstone_verification_type_small_plus +============================================================ + + [pilot] 3000 steps... +=== run_grokking_until: whetstone_verification_type_small_plus === +dataset=verification_type_train.tsv lr=0.001 wd=0.01 max_steps=3000 +Fresh start diff --git a/specialists/runs/automatability.log b/specialists/runs/automatability.log new file mode 100644 index 0000000..8a0cc4f --- /dev/null +++ b/specialists/runs/automatability.log @@ -0,0 +1,18 @@ +Training automatability_strategy specialist... + train: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/automatability_train.tsv + eval: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/automatability_eval.tsv + out: /mnt/storage/fabricate_runs/whetstone_automatability + log: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/runs/automatability_training.log + +=== run_grokking_until: whetstone_automatability === +dataset=automatability_train.tsv lr=0.001 wd=0.01 max_steps=10000 +Resuming from step 3000 +Resuming: grokking already detected at step 1000, 3 blocks since grokking. +step= 4000 overall=69.6% T=69.6% F=69.6% loss=0.0000 min=0.0000 delta=+0.0% free=904GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_automatability === +Final: step=4000 overall=69.6% +Grokking at step 1000 + +Done. Checkpoint at: /mnt/storage/fabricate_runs/whetstone_automatability/checkpoint.bin diff --git a/specialists/runs/automatability_history.json b/specialists/runs/automatability_history.json new file mode 100644 index 0000000..f98b9a7 --- /dev/null +++ b/specialists/runs/automatability_history.json @@ -0,0 +1,48 @@ +{ + "dataset": "/home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/automatability_train.tsv", + "lr": 0.001, + "weight_decay": 0.01, + "seed": "1234ABCD", + "history": [ + { + "step": 1000, + "overall": 69.56521739130434, + "true_pct": 69.56521739130434, + "false_pct": 69.56521739130434, + "delta": 19.565217391304344, + "end_loss": 0.000246, + "min_loss": 0.000246, + "grokking": true + }, + { + "step": 2000, + "overall": 69.56521739130434, + "true_pct": 69.56521739130434, + "false_pct": 69.56521739130434, + "delta": 0.0, + "end_loss": 9.2e-05, + "min_loss": 9.2e-05, + "grokking": true + }, + { + "step": 3000, + "overall": 69.56521739130434, + "true_pct": 69.56521739130434, + "false_pct": 69.56521739130434, + "delta": 0.0, + "end_loss": 4.8e-05, + "min_loss": 4.6e-05, + "grokking": true + }, + { + "step": 4000, + "overall": 69.56521739130434, + "true_pct": 69.56521739130434, + "false_pct": 69.56521739130434, + "delta": 0.0, + "end_loss": 2.7e-05, + "min_loss": 2.6e-05, + "grokking": true + } + ] +} diff --git a/specialists/runs/automatability_training.log b/specialists/runs/automatability_training.log new file mode 100644 index 0000000..23bf41d --- /dev/null +++ b/specialists/runs/automatability_training.log @@ -0,0 +1,10 @@ +=== run_grokking_until: whetstone_automatability === +dataset=automatability_train.tsv lr=0.001 wd=0.01 max_steps=10000 +Resuming from step 3000 +Resuming: grokking already detected at step 1000, 3 blocks since grokking. +step= 4000 overall=69.6% T=69.6% F=69.6% loss=0.0000 min=0.0000 delta=+0.0% free=904GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_automatability === +Final: step=4000 overall=69.6% +Grokking at step 1000 diff --git a/specialists/runs/confidence_tier_history.json b/specialists/runs/confidence_tier_history.json new file mode 100644 index 0000000..93cc82b --- /dev/null +++ b/specialists/runs/confidence_tier_history.json @@ -0,0 +1,38 @@ +{ + "dataset": "/home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/confidence_tier_train.tsv", + "lr": 0.001, + "weight_decay": 0.01, + "seed": "1234ABCD", + "history": [ + { + "step": 1000, + "overall": 100.0, + "true_pct": 100.0, + "false_pct": 100.0, + "delta": 50.0, + "end_loss": 0.000587, + "min_loss": 0.000427, + "grokking": true + }, + { + "step": 2000, + "overall": 100.0, + "true_pct": 100.0, + "false_pct": 100.0, + "delta": 0.0, + "end_loss": 0.000178, + "min_loss": 9.1e-05, + "grokking": true + }, + { + "step": 3000, + "overall": 100.0, + "true_pct": 100.0, + "false_pct": 100.0, + "delta": 0.0, + "end_loss": 5.1e-05, + "min_loss": 3e-05, + "grokking": true + } + ] +} diff --git a/specialists/runs/confidence_tier_training.log b/specialists/runs/confidence_tier_training.log new file mode 100644 index 0000000..f358a9d --- /dev/null +++ b/specialists/runs/confidence_tier_training.log @@ -0,0 +1,16 @@ +=== run_grokking_until: whetstone_confidence_tier === +dataset=confidence_tier_train.tsv lr=0.001 wd=0.01 max_steps=5000 +Fresh start +step= 1000 overall=100.0% T=100.0% F=100.0% loss=0.0006 min=0.0004 delta=+50.0% free=836GB <<< GROKKING (accuracy jump) + +============================================================ +GROKKING at step 1000! (stopping after 3 more blocks) +============================================================ + +step= 2000 overall=100.0% T=100.0% F=100.0% loss=0.0002 min=0.0001 delta=+0.0% free=834GB <<< GROKKING (loss) +step= 3000 overall=100.0% T=100.0% F=100.0% loss=0.0001 min=0.0000 delta=+0.0% free=832GB <<< GROKKING (loss) +Stopping 3 blocks after grokking (step 3000). + +=== DONE: whetstone_confidence_tier === +Final: step=3000 overall=100.0% +Grokking at step 1000 diff --git a/specialists/runs/prereq_op.log b/specialists/runs/prereq_op.log new file mode 100644 index 0000000..c448ede --- /dev/null +++ b/specialists/runs/prereq_op.log @@ -0,0 +1,18 @@ +Training prereq_op_selector specialist... + train: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/prereq_op_train.tsv + eval: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/prereq_op_eval.tsv + out: /mnt/storage/fabricate_runs/whetstone_prereq_op + log: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/runs/prereq_op_training.log + +=== run_grokking_until: whetstone_prereq_op === +dataset=prereq_op_train.tsv lr=0.001 wd=0.01 max_steps=5000 +Resuming from step 3000 +Resuming: grokking already detected at step 1000, 3 blocks since grokking. +step= 4000 overall=62.5% T=62.5% F=62.5% loss=0.0000 min=0.0000 delta=+0.0% free=904GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_prereq_op === +Final: step=4000 overall=62.5% +Grokking at step 1000 + +Done. Checkpoint at: /mnt/storage/fabricate_runs/whetstone_prereq_op/checkpoint.bin diff --git a/specialists/runs/prereq_op_combined.log b/specialists/runs/prereq_op_combined.log new file mode 100644 index 0000000..81a9c64 --- /dev/null +++ b/specialists/runs/prereq_op_combined.log @@ -0,0 +1,17 @@ +=== run_grokking_until: whetstone_prereq_op === +dataset=prereq_op_train.tsv lr=0.001 wd=0.01 max_steps=8000 +Fresh start +step= 1000 overall=69.4% T=69.4% F=69.4% loss=0.0004 min=0.0004 delta=+19.4% free=901GB <<< GROKKING (accuracy jump) + +============================================================ +GROKKING at step 1000! (stopping after 4 more blocks) +============================================================ + +step= 2000 overall=69.4% T=69.4% F=69.4% loss=0.0001 min=0.0001 delta=+0.0% free=897GB <<< GROKKING (loss) +step= 3000 overall=69.4% T=69.4% F=69.4% loss=0.0001 min=0.0001 delta=+0.0% free=893GB <<< GROKKING (loss) +step= 4000 overall=69.4% T=69.4% F=69.4% loss=0.0000 min=0.0000 delta=+0.0% free=890GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_prereq_op === +Final: step=4000 overall=69.4% +Grokking at step 1000 diff --git a/specialists/runs/prereq_op_history.json b/specialists/runs/prereq_op_history.json new file mode 100644 index 0000000..7b72ee8 --- /dev/null +++ b/specialists/runs/prereq_op_history.json @@ -0,0 +1,48 @@ +{ + "dataset": "/home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/combined/prereq_op_train.tsv", + "lr": 0.001, + "weight_decay": 0.01, + "seed": "1234ABCD", + "history": [ + { + "step": 1000, + "overall": 69.44444444444444, + "true_pct": 69.44444444444444, + "false_pct": 69.44444444444444, + "delta": 19.444444444444443, + "end_loss": 0.000413, + "min_loss": 0.0004, + "grokking": true + }, + { + "step": 2000, + "overall": 69.44444444444444, + "true_pct": 69.44444444444444, + "false_pct": 69.44444444444444, + "delta": 0.0, + "end_loss": 0.000127, + "min_loss": 0.000127, + "grokking": true + }, + { + "step": 3000, + "overall": 69.44444444444444, + "true_pct": 69.44444444444444, + "false_pct": 69.44444444444444, + "delta": 0.0, + "end_loss": 5.5e-05, + "min_loss": 5.4e-05, + "grokking": true + }, + { + "step": 4000, + "overall": 69.44444444444444, + "true_pct": 69.44444444444444, + "false_pct": 69.44444444444444, + "delta": 0.0, + "end_loss": 2.2e-05, + "min_loss": 2e-05, + "grokking": true + } + ] +} diff --git a/specialists/runs/prereq_op_training.log b/specialists/runs/prereq_op_training.log new file mode 100644 index 0000000..872602a --- /dev/null +++ b/specialists/runs/prereq_op_training.log @@ -0,0 +1,10 @@ +=== run_grokking_until: whetstone_prereq_op === +dataset=prereq_op_train.tsv lr=0.001 wd=0.01 max_steps=5000 +Resuming from step 3000 +Resuming: grokking already detected at step 1000, 3 blocks since grokking. +step= 4000 overall=62.5% T=62.5% F=62.5% loss=0.0000 min=0.0000 delta=+0.0% free=904GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_prereq_op === +Final: step=4000 overall=62.5% +Grokking at step 1000 diff --git a/specialists/runs/verif_type.log b/specialists/runs/verif_type.log new file mode 100644 index 0000000..7a3b9ac --- /dev/null +++ b/specialists/runs/verif_type.log @@ -0,0 +1,18 @@ +Training verification_type specialist... + train: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/verification_type_train.tsv + eval: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/verification_type_eval.tsv + out: /mnt/storage/fabricate_runs/whetstone_verification_type + log: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/runs/verification_type_training.log + +=== run_grokking_until: whetstone_verification_type === +dataset=verification_type_train.tsv lr=0.001 wd=0.01 max_steps=5000 +Resuming from step 3000 +Resuming: grokking already detected at step 1000, 3 blocks since grokking. +step= 4000 overall=84.9% T=84.9% F=84.9% loss=0.0000 min=0.0000 delta=+1.2% free=904GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_verification_type === +Final: step=4000 overall=84.9% +Grokking at step 1000 + +Done. Checkpoint at: /mnt/storage/fabricate_runs/whetstone_verification_type/checkpoint.bin diff --git a/specialists/runs/verification_type_history.json b/specialists/runs/verification_type_history.json new file mode 100644 index 0000000..a52ac40 --- /dev/null +++ b/specialists/runs/verification_type_history.json @@ -0,0 +1,48 @@ +{ + "dataset": "/home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/verification_type_train.tsv", + "lr": 0.001, + "weight_decay": 0.01, + "seed": "1234ABCD", + "history": [ + { + "step": 1000, + "overall": 81.97674418604652, + "true_pct": 81.97674418604652, + "false_pct": 81.97674418604652, + "delta": 31.976744186046517, + "end_loss": 0.000621, + "min_loss": 0.0006, + "grokking": true + }, + { + "step": 2000, + "overall": 83.13953488372093, + "true_pct": 83.13953488372093, + "false_pct": 83.13953488372093, + "delta": 1.16279069767441, + "end_loss": 0.000206, + "min_loss": 0.000175, + "grokking": true + }, + { + "step": 3000, + "overall": 83.72093023255815, + "true_pct": 83.72093023255815, + "false_pct": 83.72093023255815, + "delta": 0.5813953488372192, + "end_loss": 5.1e-05, + "min_loss": 5.1e-05, + "grokking": true + }, + { + "step": 4000, + "overall": 84.88372093023256, + "true_pct": 84.88372093023256, + "false_pct": 84.88372093023256, + "delta": 1.16279069767441, + "end_loss": 3.1e-05, + "min_loss": 2.5e-05, + "grokking": true + } + ] +} diff --git a/specialists/runs/verification_type_training.log b/specialists/runs/verification_type_training.log new file mode 100644 index 0000000..5d8fe41 --- /dev/null +++ b/specialists/runs/verification_type_training.log @@ -0,0 +1,10 @@ +=== run_grokking_until: whetstone_verification_type === +dataset=verification_type_train.tsv lr=0.001 wd=0.01 max_steps=5000 +Resuming from step 3000 +Resuming: grokking already detected at step 1000, 3 blocks since grokking. +step= 4000 overall=84.9% T=84.9% F=84.9% loss=0.0000 min=0.0000 delta=+1.2% free=904GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_verification_type === +Final: step=4000 overall=84.9% +Grokking at step 1000 diff --git a/specialists/runs/whetstone_automatability_large.log b/specialists/runs/whetstone_automatability_large.log new file mode 100644 index 0000000..6075014 --- /dev/null +++ b/specialists/runs/whetstone_automatability_large.log @@ -0,0 +1,44 @@ +=== whetstone_automatability (large) === + train: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/automatability_train.tsv + eval: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/automatability_eval.tsv + model: hidden=512 layers=4 heads=1 + +=== run_grokking_until: whetstone_automatability_large === +dataset=automatability_train.tsv lr=0.0005 wd=0.01 max_steps=20000 +Fresh start +Traceback (most recent call last): + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 299, in + main() + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 220, in main + block_losses = train_block( + ^^^^^^^^^^^^ + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 94, in train_block + raise RuntimeError(proc.stderr[-2000:] or proc.stdout[-2000:]) +RuntimeError: === Whetstone Fabricate === +steps=1000 batch_size=32 lr=0.000500 weight_decay=0.010000 hidden_dim=512 model=gpu_transformer morphisms=binary .morf (fast) + +[1] Loading dataset... +ProntoQADataset: 266 examples, vocab_size=140, seq_len=96 + +[2] ModelConfig + vocab_size=140 embedding_dim=512 hidden_dim=512 layers=4 heads=1 ff_hidden_dim=2048 seq_len=96 + +[3] Constructing model engine... + num_params=12795020 + morphism binary: 97.6 MB/step -> 97618.3 MB for 1000 steps + +[4] Output directory: /mnt/storage/fabricate_runs/whetstone_automatability_large +[5] --reset: removing checkpoint and output files. +[6] TrajectoryWriter: /mnt/storage/fabricate_runs/whetstone_automatability_large/train.fabr (fresh) + Initial weights written. +[7] MorphismWriter: /mnt/storage/fabricate_runs/whetstone_automatability_large/morphisms.morf (fresh) + Building fabricate.db (static tables only)... +Initialising weight_metadata (12795020 params)... + weight_metadata populated. + Static tables ready. + +[8] Training (1000 update steps from step 0, batch_size=32, lr=0.000500) + step loss task + ------ ------------ ---- + 0 5.978329 0 + diff --git a/specialists/runs/whetstone_verification_type_large.log b/specialists/runs/whetstone_verification_type_large.log new file mode 100644 index 0000000..9bcb32c --- /dev/null +++ b/specialists/runs/whetstone_verification_type_large.log @@ -0,0 +1,44 @@ +=== whetstone_verification_type (large) === + train: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/verification_type_train.tsv + eval: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/verification_type_eval.tsv + model: hidden=512 layers=4 heads=1 + +=== run_grokking_until: whetstone_verification_type_large === +dataset=verification_type_train.tsv lr=0.0005 wd=0.01 max_steps=20000 +Fresh start +Traceback (most recent call last): + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 299, in + main() + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 220, in main + block_losses = train_block( + ^^^^^^^^^^^^ + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 94, in train_block + raise RuntimeError(proc.stderr[-2000:] or proc.stdout[-2000:]) +RuntimeError: === Whetstone Fabricate === +steps=1000 batch_size=32 lr=0.000500 weight_decay=0.010000 hidden_dim=512 model=gpu_transformer morphisms=binary .morf (fast) + +[1] Loading dataset... +ProntoQADataset: 978 examples, vocab_size=1251, seq_len=96 + +[2] ModelConfig + vocab_size=1251 embedding_dim=512 hidden_dim=512 layers=4 heads=1 ff_hidden_dim=2048 seq_len=96 + +[3] Constructing model engine... + num_params=13933795 + morphism binary: 106.3 MB/step -> 106306.4 MB for 1000 steps + +[4] Output directory: /mnt/storage/fabricate_runs/whetstone_verification_type_large +[5] --reset: removing checkpoint and output files. +[6] TrajectoryWriter: /mnt/storage/fabricate_runs/whetstone_verification_type_large/train.fabr (fresh) + Initial weights written. +[7] MorphismWriter: /mnt/storage/fabricate_runs/whetstone_verification_type_large/morphisms.morf (fresh) + Building fabricate.db (static tables only)... +Initialising weight_metadata (13933795 params)... + weight_metadata populated. + Static tables ready. + +[8] Training (1000 update steps from step 0, batch_size=32, lr=0.000500) + step loss task + ------ ------------ ---- + 0 7.076128 0 + diff --git a/specialists/runs/whetstone_worker_type_large.log b/specialists/runs/whetstone_worker_type_large.log new file mode 100644 index 0000000..2b98b3a --- /dev/null +++ b/specialists/runs/whetstone_worker_type_large.log @@ -0,0 +1,44 @@ +=== whetstone_worker_type (large) === + train: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/worker_type_train.tsv + eval: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/worker_type_eval.tsv + model: hidden=512 layers=4 heads=1 + +=== run_grokking_until: whetstone_worker_type_large === +dataset=worker_type_train.tsv lr=0.0005 wd=0.01 max_steps=20000 +Fresh start +Traceback (most recent call last): + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 299, in + main() + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 220, in main + block_losses = train_block( + ^^^^^^^^^^^^ + File "/home/bill/Documents/WhetstoneAI_Fabricate/run_grokking_until.py", line 94, in train_block + raise RuntimeError(proc.stderr[-2000:] or proc.stdout[-2000:]) +RuntimeError: === Whetstone Fabricate === +steps=1000 batch_size=32 lr=0.000500 weight_decay=0.010000 hidden_dim=512 model=gpu_transformer morphisms=binary .morf (fast) + +[1] Loading dataset... +ProntoQADataset: 300 examples, vocab_size=539, seq_len=96 + +[2] ModelConfig + vocab_size=539 embedding_dim=512 hidden_dim=512 layers=4 heads=1 ff_hidden_dim=2048 seq_len=96 + +[3] Constructing model engine... + num_params=13203995 + morphism binary: 100.7 MB/step -> 100738.5 MB for 1000 steps + +[4] Output directory: /mnt/storage/fabricate_runs/whetstone_worker_type_large +[5] --reset: removing checkpoint and output files. +[6] TrajectoryWriter: /mnt/storage/fabricate_runs/whetstone_worker_type_large/train.fabr (fresh) + Initial weights written. +[7] MorphismWriter: /mnt/storage/fabricate_runs/whetstone_worker_type_large/morphisms.morf (fresh) + Building fabricate.db (static tables only)... +Initialising weight_metadata (13203995 params)... + weight_metadata populated. + Static tables ready. + +[8] Training (1000 update steps from step 0, batch_size=32, lr=0.000500) + step loss task + ------ ------------ ---- + 0 6.984013 0 + diff --git a/specialists/runs/worker_type.log b/specialists/runs/worker_type.log new file mode 100644 index 0000000..d4124f6 --- /dev/null +++ b/specialists/runs/worker_type.log @@ -0,0 +1,18 @@ +Training worker_type specialist... + train: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/worker_type_train.tsv + eval: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/worker_type_eval.tsv + out: /mnt/storage/fabricate_runs/whetstone_worker_type + log: /home/bill/Documents/CLionProjects/whetstone_DSL/specialists/runs/worker_type_training.log + +=== run_grokking_until: whetstone_worker_type === +dataset=worker_type_train.tsv lr=0.001 wd=0.01 max_steps=5000 +Resuming from step 3000 +Resuming: grokking already detected at step 1000, 3 blocks since grokking. +step= 4000 overall=88.5% T=88.5% F=88.5% loss=0.0000 min=0.0000 delta=+7.7% free=904GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_worker_type === +Final: step=4000 overall=88.5% +Grokking at step 1000 + +Done. Checkpoint at: /mnt/storage/fabricate_runs/whetstone_worker_type/checkpoint.bin diff --git a/specialists/runs/worker_type_history.json b/specialists/runs/worker_type_history.json new file mode 100644 index 0000000..a329bcc --- /dev/null +++ b/specialists/runs/worker_type_history.json @@ -0,0 +1,48 @@ +{ + "dataset": "/home/bill/Documents/CLionProjects/whetstone_DSL/specialists/data/generated/worker_type_train.tsv", + "lr": 0.001, + "weight_decay": 0.01, + "seed": "1234ABCD", + "history": [ + { + "step": 1000, + "overall": 80.76923076923077, + "true_pct": 80.76923076923077, + "false_pct": 80.76923076923077, + "delta": 30.769230769230774, + "end_loss": 0.0004, + "min_loss": 0.000391, + "grokking": true + }, + { + "step": 2000, + "overall": 80.76923076923077, + "true_pct": 80.76923076923077, + "false_pct": 80.76923076923077, + "delta": 0.0, + "end_loss": 0.000126, + "min_loss": 0.000124, + "grokking": true + }, + { + "step": 3000, + "overall": 80.76923076923077, + "true_pct": 80.76923076923077, + "false_pct": 80.76923076923077, + "delta": 0.0, + "end_loss": 4.7e-05, + "min_loss": 4.7e-05, + "grokking": true + }, + { + "step": 4000, + "overall": 88.46153846153845, + "true_pct": 88.46153846153845, + "false_pct": 88.46153846153845, + "delta": 7.692307692307679, + "end_loss": 2.1e-05, + "min_loss": 2e-05, + "grokking": true + } + ] +} diff --git a/specialists/runs/worker_type_training.log b/specialists/runs/worker_type_training.log new file mode 100644 index 0000000..2c18ebb --- /dev/null +++ b/specialists/runs/worker_type_training.log @@ -0,0 +1,10 @@ +=== run_grokking_until: whetstone_worker_type === +dataset=worker_type_train.tsv lr=0.001 wd=0.01 max_steps=5000 +Resuming from step 3000 +Resuming: grokking already detected at step 1000, 3 blocks since grokking. +step= 4000 overall=88.5% T=88.5% F=88.5% loss=0.0000 min=0.0000 delta=+7.7% free=904GB <<< GROKKING (loss) +Stopping 4 blocks after grokking (step 4000). + +=== DONE: whetstone_worker_type === +Final: step=4000 overall=88.5% +Grokking at step 1000 diff --git a/specialists/scripts/capacity_sweep.py b/specialists/scripts/capacity_sweep.py new file mode 100644 index 0000000..85ba3c4 --- /dev/null +++ b/specialists/scripts/capacity_sweep.py @@ -0,0 +1,288 @@ +#!/usr/bin/env python3 +""" +capacity_sweep.py + +Runs a staged capacity sweep for one specialist gate. +Tiers: small+ (256/2) → medium (384/3) → large (512/4). + +For each tier: + 1. Run a short pilot (3000 steps). If accuracy doesn't improve over + the previous tier's result, skip the full run. + 2. Run the full training (up to --max-steps) only if pilot improved. + 3. Evaluate with eval_specialist.py (confusion, confidence, thresholds). + 4. Append result row to the sweep table. + +Usage: + python3 specialists/scripts/capacity_sweep.py \\ + --name worker_type \\ + --train specialists/data/generated/worker_type_train.tsv \\ + --eval specialists/data/generated/worker_type_eval.tsv \\ + --labels "implementer,reviewer,architect,qa" \\ + --runs-dir /mnt/storage/fabricate_runs \\ + --results-out specialists/eval/results/worker_type_sweep.json + + # Run from whetstone_DSL root. +""" + +import argparse +import json +import os +import shutil +import subprocess +import sys +import time +from pathlib import Path + +ROOT = Path(__file__).parent.parent.parent # whetstone_DSL root +FABRICATE = Path("/home/bill/Documents/WhetstoneAI_Fabricate") +VENV_PYTHON = FABRICATE / ".venv" / "bin" / "python3" +TRAIN_SCRIPT = ROOT / "specialists" / "scripts" / "train_specialist_pt.py" +EVAL_SCRIPT = ROOT / "specialists" / "eval" / "eval_specialist_pt.py" + +TIERS = [ + {"name": "small_plus", "hidden": 256, "layers": 2, "heads": 4}, + {"name": "medium", "hidden": 384, "layers": 3, "heads": 6}, + {"name": "large", "hidden": 512, "layers": 4, "heads": 8}, +] + +PILOT_STEPS = 3000 # steps for pilot run +FULL_STEPS = 15000 # max steps for full run +BLOCK = 1000 # steps per eval block +BATCH_SIZE = 256 +LR = 0.001 +WD = 0.01 + + +def log(msg): + print(msg, flush=True) + + +def run_training(run_dir: Path, train_data: Path, eval_data: Path, + labels: str, hidden: int, layers: int, heads: int, + max_steps: int, reset: bool, history_out: Path) -> bool: + """Invoke train_specialist_pt.py. Returns True on success.""" + run_dir.mkdir(parents=True, exist_ok=True) + if reset: + for f in run_dir.glob("*"): + f.unlink() + history_out.unlink(missing_ok=True) + + cmd = [ + str(VENV_PYTHON), str(TRAIN_SCRIPT), + "--dataset", str(train_data), + "--eval-dataset", str(eval_data), + "--out-dir", str(run_dir), + "--labels", labels, + "--lr", str(LR), + "--weight-decay", str(WD), + "--max-steps", str(max_steps), + "--hidden-dim", str(hidden), + "--layers", str(layers), + "--heads", str(heads), + "--batch-size", str(BATCH_SIZE), + "--grok-loss-threshold", "0.05", + "--grok-acc-jump", "10.0", + "--stop-after-grokking-blocks", "4", + "--history-out", str(history_out), + "--reset", + ] + + result = subprocess.run(cmd, capture_output=False) + return result.returncode == 0 + + +def run_eval(run_dir: Path, eval_data: Path, labels: str) -> dict | None: + """Run eval_specialist_pt.py on the checkpoint. Returns parsed JSON or None.""" + ck = run_dir / "checkpoint.pt" + if not ck.exists(): + return None + + result = subprocess.run( + [str(VENV_PYTHON), str(EVAL_SCRIPT), + "--checkpoint", str(ck), + "--dataset", str(eval_data), + "--labels", labels, + "--quiet"], + capture_output=True, text=True, + ) + if result.returncode != 0: + log(f" [WARN] eval failed: {result.stderr[-500:]}") + return None + try: + # Last line should be the JSON dump + last_line = result.stdout.strip().splitlines()[-1] + return json.loads(last_line) + except (json.JSONDecodeError, IndexError): + log(f" [WARN] could not parse eval JSON") + return None + + +def read_history(history_path: Path) -> dict: + if not history_path.exists(): + return {} + return json.loads(history_path.read_text()) + + +def get_best_accuracy(history: dict) -> float: + entries = history.get("history", []) + if not entries: + return 0.0 + return max(e.get("overall", 0.0) for e in entries) + + +def suggest_action(eval_result: dict, prev_accuracy: float) -> str: + if eval_result is None: + return "training_failed" + acc = eval_result.get("overall_accuracy", 0.0) + verdict = eval_result.get("verdict", "unknown") + + delta = acc - prev_accuracy + if delta < 1.0: + return "no_gain_stop_scaling" + return verdict + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--name", required=True, help="Gate name, e.g. worker_type") + ap.add_argument("--train", required=True, help="Training TSV path") + ap.add_argument("--eval", required=True, help="Eval TSV path") + ap.add_argument("--labels", required=True, help="Comma-sep label tokens") + ap.add_argument("--runs-dir", default="/mnt/storage/fabricate_runs") + ap.add_argument("--results-out", default=None, help="JSON output path") + ap.add_argument("--tiers", default="small_plus,medium,large", + help="Comma-sep tier names to run (subset of small_plus,medium,large)") + ap.add_argument("--pilot-only", action="store_true", + help="Only run pilots, no full training runs") + args = ap.parse_args() + + train_data = Path(args.train) + eval_data = Path(args.eval) + runs_base = Path(args.runs_dir) + active_tiers = [t for t in TIERS if t["name"] in args.tiers.split(",")] + + sweep_results = { + "gate": args.name, + "labels": args.labels, + "train": str(train_data), + "eval": str(eval_data), + "tiers": [], + } + + prev_accuracy = 0.0 # baseline for gain comparison + results_dir = ROOT / "specialists" / "eval" / "results" + results_dir.mkdir(parents=True, exist_ok=True) + + for tier in active_tiers: + tname = tier["name"] + hidden = tier["hidden"] + layers = tier["layers"] + heads = tier.get("heads", 4) + run_id = f"whetstone_{args.name}_{tname}" + run_dir = runs_base / run_id + history = run_dir / "history.json" + tier_result_path = results_dir / f"{args.name}_{tname}.json" + + log(f"\n{'='*60}") + log(f" Tier: {tname} hidden={hidden} layers={layers} heads={heads}") + log(f" Run dir: {run_dir}") + log(f"{'='*60}") + + tier_row = { + "tier": tname, + "hidden": hidden, + "layers": layers, + "heads": heads, + "model_size_approx": f"~{hidden*hidden*layers*12 // 1_000_000}M params", + } + + # --- Pilot run --- + log(f"\n [pilot] {PILOT_STEPS} steps...") + t0 = time.perf_counter() + ok = run_training(run_dir, train_data, eval_data, args.labels, + hidden, layers, heads, PILOT_STEPS, reset=True, history_out=history) + pilot_time = time.perf_counter() - t0 + pilot_history = read_history(history) + pilot_acc = get_best_accuracy(pilot_history) + + log(f" [pilot] done in {pilot_time:.0f}s best={pilot_acc:.1f}%") + + if not ok or pilot_acc == 0.0: + log(f" [pilot] FAILED or no accuracy recorded — skipping tier") + tier_row["pilot_accuracy"] = None + tier_row["action"] = "pilot_failed" + sweep_results["tiers"].append(tier_row) + continue + + tier_row["pilot_accuracy"] = round(pilot_acc, 1) + tier_row["pilot_time_s"] = round(pilot_time, 0) + + gain = pilot_acc - prev_accuracy + log(f" [pilot] gain vs prev tier: {gain:+.1f}pp") + + if gain < 1.0 and prev_accuracy > 0: + log(f" [pilot] <1pp gain — skipping full run for this tier") + tier_row["action"] = "no_gain_stop_scaling" + sweep_results["tiers"].append(tier_row) + break + + if args.pilot_only: + log(f" [pilot-only mode] evaluating pilot checkpoint...") + else: + # --- Full run --- + log(f"\n [full] continuing to {FULL_STEPS} steps...") + t1 = time.perf_counter() + run_training(run_dir, train_data, eval_data, args.labels, + hidden, layers, heads, FULL_STEPS, reset=False, history_out=history) + full_time = time.perf_counter() - t1 + full_history = read_history(history) + full_acc = get_best_accuracy(full_history) + log(f" [full] done in {full_time:.0f}s best={full_acc:.1f}%") + tier_row["full_accuracy"] = round(full_acc, 1) + tier_row["full_time_s"] = round(full_time, 0) + + # --- Eval --- + log(f"\n [eval] running extended evaluation...") + eval_result = run_eval(run_dir, eval_data, args.labels) + if eval_result: + # Save per-tier eval result + tier_result_path.write_text(json.dumps(eval_result, indent=2)) + log(f" [eval] accuracy={eval_result['overall_accuracy']}% " + f"verdict={eval_result['verdict']}") + + tier_row["eval_accuracy"] = eval_result["overall_accuracy"] + tier_row["per_class"] = eval_result.get("per_class", {}) + tier_row["confusion_summary"] = eval_result.get("confusion_summary", [])[:5] + tier_row["threshold_analysis"] = eval_result.get("threshold_analysis", []) + tier_row["guardrail_threshold"] = eval_result.get("guardrail_threshold") + tier_row["guardrail_abstain_rate"] = eval_result.get("guardrail_abstain_rate") + tier_row["eval_latency_ms"] = eval_result.get("eval_latency_ms_per_example") + tier_row["verdict"] = eval_result["verdict"] + tier_row["action"] = suggest_action(eval_result, prev_accuracy) + tier_row["eval_result_path"] = str(tier_result_path) + + prev_accuracy = eval_result["overall_accuracy"] + else: + tier_row["action"] = "eval_failed" + + sweep_results["tiers"].append(tier_row) + + # --- Final table --- + log(f"\n{'='*60}") + log(f" SWEEP RESULTS: {args.name}") + log(f"{'='*60}") + header = f" {'tier':12s} {'hidden/L':8s} {'pilot':7s} {'full':7s} {'verdict'}" + log(header) + for row in sweep_results["tiers"]: + pilot = f"{row.get('pilot_accuracy','—'):>5}" if row.get('pilot_accuracy') else " — " + full = f"{row.get('eval_accuracy','—'):>5}" if row.get('eval_accuracy') else " — " + hl = f"{row['hidden']}/{row['layers']}" + log(f" {row['tier']:12s} {hl:8s} {pilot}% {full}% {row.get('verdict','—')}") + + out = args.results_out or str(results_dir / f"{args.name}_sweep.json") + Path(out).write_text(json.dumps(sweep_results, indent=2)) + log(f"\nFull results → {out}") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/gen_automatability_data.py b/specialists/scripts/gen_automatability_data.py new file mode 100644 index 0000000..91aa048 --- /dev/null +++ b/specialists/scripts/gen_automatability_data.py @@ -0,0 +1,293 @@ +#!/usr/bin/env python3 +""" +gen_automatability_data.py + +Labels the editor/src/*.h headers to produce training data for the +automatability_strategy specialist. + +The six tiers (matching AutomatabilityAnnotation.strategy in Annotation.h): + + deterministic — Same input always produces same output. Pure formula, + rule engine, or algorithmic transform. Zero parameters. + Examples: validators, scorers, parsers, extractors, trackers. + + template — Deterministic composition of known structural patterns. + Code generators driven by fixed templates; no ambiguity. + Examples: code generators, builders, highlighters, register tools. + + specialist — Bounded output vocabulary (fixed N classes). Probabilistic + but constrained. ~800KB model, microseconds. + Examples: selectors with uncertainty, judges, advisors, + annotators that make probabilistic choices. + + slm — Open vocabulary but local-scale. Complex transformation + or interpretation where output tokens vary but scope is + bounded. ~1-7GB. + Examples: transpilers, type translators, optimizers. + + llm — Full generative reasoning. Cross-context, novel output, + planning-level decisions. + Examples: architect intake, decomposers, agent code gen, + high-level orchestrators. + + human — Not yet formalizable. Requires manual judgment, approval, + or editorial oversight. + Examples: mutation approval, review surfaces, escalation. + +Input text format (what the specialist sees at inference time): + "name=ClassName" + +This mirrors what Claude provides when calling add_skeleton_node — the +class name is the primary signal. The Whetstone naming convention +enforces semantic suffixes, making the name a strong predictor. + +Output: TSV with columns: labelhopstext + label: integer index 0–5 + 0=deterministic 1=template 2=specialist 3=slm 4=llm 5=human + hops: always 0 + text: "name=ClassName" + +Usage: + python3 specialists/scripts/gen_automatability_data.py \\ + --src-dir editor/src \\ + --train-out specialists/data/generated/automatability_train.tsv \\ + --eval-out specialists/data/generated/automatability_eval.tsv +""" +import re +import sys +import random +import argparse +from pathlib import Path +from collections import Counter + +# --------------------------------------------------------------------------- +# Classification heuristics — suffix and prefix patterns on class names +# --------------------------------------------------------------------------- +# +# Priority order: human > llm > slm > specialist > template > deterministic +# (more specific / rarer tiers checked first to avoid being swallowed by +# the large deterministic default) + +# human: requires manual judgment, approval workflows +HUMAN_SUFFIXES = ( + "Approval", "ReviewInterface", "ReviewSurface", "MutationPreview", + "EscalationPlanner", "ApprovalGate", +) +HUMAN_SUBSTRINGS = ( + "Approval", "HumanReview", "ManualGate", "ReviewBoard", +) + +# llm: high-level reasoning, planning, novel generation +LLM_PREFIXES = ( + "Agent", # AgentCodeGen, AgentMutationApproval (covered by human above) +) +LLM_SUFFIXES = ( + "IntakeProcessor", "Decomposer", "ProjectPipeline", "ModuleDecomposer", + "ProblemParser", "MultiLanguageOrchestrator", "WorkflowOrchestrator", + "PolyglotOrchestrator", "PolyglotSuiteOrchestrator", + "TechStackSelector", # open-ended tech selection = llm (vs bounded = specialist) + "ScaffoldGenerator", # spec-to-scaffold = llm-driven +) +LLM_SUBSTRINGS = ( + "Planner", "Strategist", +) + +# slm: complex transformation, open output vocab but local scope +SLM_SUFFIXES = ( + "Transpiler", "TypeSystemTranslator", "TypeAwareMappings", + "TransformEngine", "TransformEngineExtended", + "Optimizer", "Lowering", +) +SLM_SUBSTRINGS = ( + "Transpil", "Transform", + "Translat", # TranslationReport is excluded below via DETERMINISTIC_OVERRIDES +) + +# Names that would otherwise match a higher tier but are actually deterministic. +# Checked before any tier classification. +DETERMINISTIC_OVERRIDES = { + "TranslationReport", # report doc, not a transformer + "DWARFBoundaryAnnotator", # rule-based DWARF annotation + "IntakeTextUtil", # text utility, not an intake processor +} + +# specialist: bounded probabilistic — selectors, judges, advisors, annotators +SPECIALIST_SUFFIXES = ( + "Selector", "Judge", "Advisor", "Recommender", + "Annotator", "Predictor", "Classifier", +) +SPECIALIST_SUBSTRINGS = ( + "Selector", "Judge", "Advisor", +) + +# template: code/output generation from fixed patterns +TEMPLATE_PREFIXES = ( + "Register", # RegisterXxxTools.h — tool registration templates + "SyntaxHighlighter", +) +TEMPLATE_SUFFIXES = ( + "Generator", "Builder", "Factory", "Templates", "Highlighter", + "Emitter", "Renderer", "Serializer", "Encoder", "Composer", + "Formatter", # output formatters = deterministic, but code formatters = template + "CodeGen", +) +TEMPLATE_SUBSTRINGS = ( + "Generator", "Builder", "Templates", +) + +# deterministic: pure computation — everything else +# (validators, scorers, parsers, extractors, trackers, monitors, etc.) + + +LABEL_ORDER = ["deterministic", "template", "specialist", "slm", "llm", "human"] +LABEL_INDEX = {lbl: i for i, lbl in enumerate(LABEL_ORDER)} + + +def classify(class_name: str) -> str: + n = class_name # keep original case for prefix/suffix matching + + # Hard overrides — names that pattern-match a higher tier but are deterministic + if n in DETERMINISTIC_OVERRIDES: + return "deterministic" + + # human — must check before llm (some overlap on Agent* names) + for suf in HUMAN_SUFFIXES: + if n.endswith(suf): + return "human" + for sub in HUMAN_SUBSTRINGS: + if sub in n: + return "human" + + # llm — planning-level, open-ended + for pre in LLM_PREFIXES: + if n.startswith(pre): + return "llm" + for suf in LLM_SUFFIXES: + if n.endswith(suf): + return "llm" + for sub in LLM_SUBSTRINGS: + if sub in n: + return "llm" + + # slm — complex transformation + for suf in SLM_SUFFIXES: + if n.endswith(suf): + return "slm" + for sub in SLM_SUBSTRINGS: + if sub in n: + return "slm" + + # specialist — bounded probabilistic decision + for suf in SPECIALIST_SUFFIXES: + if n.endswith(suf): + return "specialist" + for sub in SPECIALIST_SUBSTRINGS: + if sub in n: + return "specialist" + + # template — code/output generation + for pre in TEMPLATE_PREFIXES: + if n.startswith(pre): + return "template" + for suf in TEMPLATE_SUFFIXES: + if n.endswith(suf): + return "template" + for sub in TEMPLATE_SUBSTRINGS: + if sub in n: + return "template" + + # deterministic — default + return "deterministic" + + +def header_to_class_name(path: Path) -> str: + """Convert FileName.h → FileName (strip extension).""" + return path.stem + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--src-dir", default="editor/src", + help="Directory containing *.h headers (non-recursive)") + ap.add_argument("--train-out", default="specialists/data/generated/automatability_train.tsv") + ap.add_argument("--eval-out", default="specialists/data/generated/automatability_eval.tsv") + ap.add_argument("--eval-frac", type=float, default=0.15) + ap.add_argument("--seed", type=int, default=42) + ap.add_argument("--show-labels", action="store_true", + help="Print each header and its assigned label (for review)") + args = ap.parse_args() + + src_dir = Path(args.src_dir) + headers = sorted(src_dir.glob("*.h")) + if not headers: + print(f"No .h files found in {src_dir}", file=sys.stderr) + sys.exit(1) + print(f"Found {len(headers)} header files") + + all_examples: list[tuple[str, str]] = [] + for h in headers: + class_name = header_to_class_name(h) + label = classify(class_name) + all_examples.append((label, class_name)) + if args.show_labels: + print(f" {label:15s} {class_name}") + + counts = Counter(lbl for lbl, _ in all_examples) + print("Raw class distribution:", dict(counts)) + + # Balance: use the median class size as target so we keep more deterministic + # examples (the dominant real-world case) while still oversampling sparse tiers. + sorted_counts = sorted(counts.values()) + median_count = sorted_counts[len(sorted_counts) // 2] + target = min(median_count * 2, max(sorted_counts)) + target = max(target, 20) + print(f"Balancing to {target} examples per class") + + rng = random.Random(args.seed) + by_class: dict[str, list[str]] = {} + for lbl, name in all_examples: + by_class.setdefault(lbl, []).append(name) + + balanced: list[tuple[str, str]] = [] + for lbl in LABEL_ORDER: + names = by_class.get(lbl, []) + if not names: + print(f"WARNING: no examples for class '{lbl}'", file=sys.stderr) + continue + rng.shuffle(names) + chosen = names[:target] + while len(chosen) < target: + chosen += names + balanced.extend((lbl, name) for name in chosen[:target]) + + rng.shuffle(balanced) + print(f"Balanced total: {len(balanced)} examples") + + n_eval = max(1, int(len(balanced) * args.eval_frac)) + eval_set = balanced[:n_eval] + train_set = balanced[n_eval:] + + Path(args.train_out).parent.mkdir(parents=True, exist_ok=True) + Path(args.eval_out).parent.mkdir(parents=True, exist_ok=True) + + def write_tsv(path, rows): + with open(path, "w") as f: + for lbl, name in rows: + idx = LABEL_INDEX[lbl] + f.write(f"{idx}\t0\tname={name}\n") + print(f"Wrote {len(rows)} rows → {path}") + + write_tsv(args.train_out, train_set) + write_tsv(args.eval_out, eval_set) + + for split_name, split in [("train", train_set), ("eval", eval_set)]: + c = Counter(lbl for lbl, _ in split) + print(f" {split_name}: {dict(sorted(c.items()))}") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/gen_confidence_tier_data.py b/specialists/scripts/gen_confidence_tier_data.py new file mode 100644 index 0000000..7053459 --- /dev/null +++ b/specialists/scripts/gen_confidence_tier_data.py @@ -0,0 +1,130 @@ +#!/usr/bin/env python3 +""" +gen_confidence_tier_data.py + +Generate synthetic training data for the confidence_tier specialist. + +The Whetstone confidence formula (from TaskitemConfidenceAmbiguity.h): + base = 85 + - 25 if not queueReady + - 10 per ambiguous requirement (up to 3) + - 8 per conflict detected + - 5 if prereq_count > 2 + - 4 if has_dependencies + clamped to [0, 100] + +Tier labels: + 0 = high (confidence >= 80) + 1 = medium (confidence 60–79) + 2 = low (confidence < 60) + +The specialist encodes features as a short token sequence: + "conflicts=N ambiguity=M deps=D prereqs=P queueready=B" + +Usage: + python3 specialists/scripts/gen_confidence_tier_data.py \ + --train-out specialists/data/generated/confidence_tier_train.tsv \ + --eval-out specialists/data/generated/confidence_tier_eval.tsv +""" +import random +import argparse +from pathlib import Path +from collections import Counter + +LABEL_ORDER = ["high", "medium", "low"] +LABEL_INDEX = {lbl: i for i, lbl in enumerate(LABEL_ORDER)} + + +def compute_confidence(conflicts, ambiguity, deps, prereqs, queue_ready): + conf = 85 + if not queue_ready: + conf -= 25 + conf -= min(ambiguity, 3) * 10 + conf -= conflicts * 8 + if prereqs > 2: + conf -= 5 + if deps > 0: + conf -= 4 + return max(0, min(100, conf)) + + +def tier(conf): + if conf >= 80: + return "high" + elif conf >= 60: + return "medium" + else: + return "low" + + +def encode_input(conflicts, ambiguity, deps, prereqs, queue_ready): + qr = "yes" if queue_ready else "no" + return f"conflicts={conflicts} ambiguity={ambiguity} deps={deps} prereqs={prereqs} queueready={qr}" + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--train-out", default="specialists/data/generated/confidence_tier_train.tsv") + ap.add_argument("--eval-out", default="specialists/data/generated/confidence_tier_eval.tsv") + ap.add_argument("--n-per-class", type=int, default=400) + ap.add_argument("--seed", type=int, default=42) + args = ap.parse_args() + + rng = random.Random(args.seed) + + # Generate all unique input combinations and label them + seen = set() + by_tier: dict[str, list[str]] = {"high": [], "medium": [], "low": []} + + for conflicts in range(0, 10): + for ambiguity in range(0, 4): + for deps in range(0, 3): + for prereqs in range(1, 5): + for queue_ready in (True, False): + key = (conflicts, ambiguity, deps, prereqs, queue_ready) + if key in seen: + continue + seen.add(key) + conf = compute_confidence(*key) + t = tier(conf) + text = encode_input(*key) + by_tier[t].append(text) + + print("Raw tier distribution:", {k: len(v) for k, v in by_tier.items()}) + + # Balance to n_per_class (oversample if needed) + target = args.n_per_class + balanced = [] + for lbl, examples in by_tier.items(): + rng.shuffle(examples) + chosen = examples[:] + while len(chosen) < target: + chosen.extend(examples) + chosen = chosen[:target] + balanced.extend((lbl, ex) for ex in chosen) + + rng.shuffle(balanced) + + n_eval = max(1, int(len(balanced) * 0.15)) + eval_set = balanced[:n_eval] + train_set = balanced[n_eval:] + + Path(args.train_out).parent.mkdir(parents=True, exist_ok=True) + Path(args.eval_out).parent.mkdir(parents=True, exist_ok=True) + + def write_tsv(path, rows): + with open(path, "w") as f: + for lbl, text in rows: + f.write(f"{LABEL_INDEX[lbl]}\t0\t{text}\n") + print(f"Wrote {len(rows)} rows → {path}") + + write_tsv(args.train_out, train_set) + write_tsv(args.eval_out, eval_set) + + for split_name, split in [("train", train_set), ("eval", eval_set)]: + c = Counter(lbl for lbl, _ in split) + print(f" {split_name}: {dict(sorted(c.items()))}") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/gen_from_projects.py b/specialists/scripts/gen_from_projects.py new file mode 100644 index 0000000..6fb6b31 --- /dev/null +++ b/specialists/scripts/gen_from_projects.py @@ -0,0 +1,332 @@ +#!/usr/bin/env python3 +""" +gen_from_projects.py + +Runs all project benchmark specs through the whetstone_mcp pipeline +(architect_intake → generate_taskitems) and extracts labeled training +examples for each specialist from the real pipeline output. + +Why this matters: + - Training data from sprint plans uses step description text. + - Specialists run at inference time on taskitem TITLES + INTENT. + - These are similar but not identical formats. Running real specs through + the pipeline produces training examples in the exact inference-time format. + - The pipeline output is the "correct answer" — workerType from inferWorkerType, + prerequisiteOps from inferPrerequisiteOps, verificationType from the contract. + +Input files: + datasets/project_benchmarks/common_projects_100.jsonl (100 projects) + datasets/project_benchmarks/challenging_*.jsonl (25 projects) + +Output TSVs (appended to existing data or written fresh): + specialists/data/generated/prereq_op_projects_train.tsv + specialists/data/generated/prereq_op_projects_eval.tsv + specialists/data/generated/worker_type_projects_train.tsv + specialists/data/generated/worker_type_projects_eval.tsv + specialists/data/generated/verification_type_projects_train.tsv + specialists/data/generated/verification_type_projects_eval.tsv + +Text format for all specialists: + " | <intent[:120]>" + Using pipe separator so title and intent are both visible but the + model sees them as one continuous text sequence. + +Usage: + python3 specialists/scripts/gen_from_projects.py \\ + --whetstone-bin editor/build-native/whetstone_mcp \\ + --workspace . +""" +import argparse +import json +import random +import subprocess +import sys +from collections import Counter +from pathlib import Path + +# --------------------------------------------------------------------------- +# Label maps +# --------------------------------------------------------------------------- + +# Text-based prereq classifier — same tokens as gen_prereq_op_data.py so that +# project-spec task text is labeled consistently with the sprint-plan data. +# Using text rather than the C++ heuristic's prerequisiteOps because the heuristic +# never emits "manual-approval" (circular labels = all "standard"). +_PREREQ_REVIEW_TOKENS = { + "introduce", "phase", "framework", "architecture", "design", "refactor", + "planning", "taxonomy", "grammar", "interface", "api surface", "subsystem", + "extension point", "abstraction", "decompos", "scaffold", "polyglot", + "cross-language", "cross-project", "multi-language", "orchestrat", + "migration plan", "phase plan", "protocol design", "new layer", + "foundation", "bootstrap", "initial", "phase 1", "phase 2", "phase 3", + "sprint plan", "roadmap", +} +_PREREQ_APPROVAL_TOKENS = { + "release", "deploy", "publish", "migrate", "security", "compliance", + "upgrade", "breaking change", "production", "rollout", "finalize", + "sign off", "certification", "approval", "cutover", "go-live", + "data migration", "schema migration", "breaking", "deprecat", + "audit", "regulatory", "gdpr", "pii", "secret", "credential", "vault", + "access control", "rbac", "acl", "permission", "fraud", "encryption", +} + +def prereq_ops_to_class(ops: list[str]) -> str: + """Legacy: classify by actual ops list (unused — text_to_prereq_class is preferred).""" + s = set(ops) + has_review = "architect-review" in s + has_approval = "manual-approval" in s + if has_review and has_approval: + return "full_gates" + if has_review: + return "needs_review" + if has_approval: + return "needs_approval" + return "standard" + +def text_to_prereq_class(text: str) -> str: + """Label prereq class from task title+intent text (same signal the specialist sees).""" + t = text.lower() + has_review = any(tok in t for tok in _PREREQ_REVIEW_TOKENS) + has_approval = any(tok in t for tok in _PREREQ_APPROVAL_TOKENS) + if has_review and has_approval: + return "full_gates" + if has_review: + return "needs_review" + if has_approval: + return "needs_approval" + return "standard" + +PREREQ_LABEL_INDEX = {"standard": 0, "needs_review": 1, "needs_approval": 2, "full_gates": 3} +WORKER_LABEL_INDEX = {"implementer": 0, "reviewer": 1, "architect": 2, "qa": 3} +VERIF_LABEL_INDEX = {"unit": 0, "integration": 1, "schema": 2, "smoke": 3, "docs": 4} + +# --------------------------------------------------------------------------- +# MCP pipeline helpers +# --------------------------------------------------------------------------- + +def mcp_call(binary: str, workspace: str, tool: str, args: dict) -> dict: + payload = json.dumps({ + "jsonrpc": "2.0", "id": 1, + "method": "tools/call", + "params": {"name": tool, "arguments": args} + }) + try: + r = subprocess.run( + [binary, "--workspace", workspace, "--language", "cpp"], + input=payload, capture_output=True, text=True, timeout=20 + ) + content = (json.loads(r.stdout) + .get("result", {}) + .get("content", [{}])[0] + .get("text", "")) + return json.loads(content) if content else {} + except Exception as e: + print(f" [WARN] mcp_call {tool}: {e}", file=sys.stderr) + return {} + + +def run_pipeline(binary: str, workspace: str, spec: str) -> list[dict]: + """Run architect_intake → generate_taskitems for one spec. Return task list.""" + intake = mcp_call(binary, workspace, "whetstone_architect_intake", {"markdown": spec}) + reqs = intake.get("normalizedRequirements", []) + if not reqs: + return [] + result = mcp_call(binary, workspace, "whetstone_generate_taskitems", { + "normalizedRequirements": reqs, + "conflicts": intake.get("conflicts", []) + }) + return result.get("tasks", []) + + +def task_text(task: dict) -> str: + """Combine title + intent for specialist input.""" + title = task.get("title", "").strip() + intent = task.get("intent", "").strip()[:120] + if intent and intent != title: + return f"{title} | {intent}" + return title + + +# --------------------------------------------------------------------------- +# Load project specs +# --------------------------------------------------------------------------- + +def load_projects(dataset_dir: Path) -> list[dict]: + projects = [] + # Common 100 + common = dataset_dir / "common_projects_100.jsonl" + if common.exists(): + with open(common) as f: + for line in f: + projects.append(json.loads(line)) + # All challenging sets + for path in sorted(dataset_dir.glob("challenging_*.jsonl")): + with open(path) as f: + for line in f: + p = json.loads(line) + if p.get("spec"): + projects.append(p) + print(f"Loaded {len(projects)} project specs") + return projects + + +def spec_markdown(project: dict) -> str: + """Wrap a project spec into the sections architect_intake expects.""" + return ( + f"## Goals\n{project['spec']}\n\n" + f"## Constraints\n- Production quality\n- Well-tested\n\n" + f"## Acceptance Criteria\n- All core features implemented\n" + f"- Tests passing\n" + ) + + +# --------------------------------------------------------------------------- +# Balance + write TSV +# --------------------------------------------------------------------------- + +def balance_and_split(examples: list[tuple[str, str]], + label_index: dict[str, int], + label_order: list[str], + seed: int = 42, + eval_frac: float = 0.15, + min_per_class: int = 20) -> tuple[list, list]: + """Return (train_rows, eval_rows) as list of (label_str, text) pairs.""" + by_class: dict[str, list[str]] = {} + for lbl, text in examples: + by_class.setdefault(lbl, []).append(text) + + counts = {lbl: len(items) for lbl, items in by_class.items()} + print(f" Raw class distribution: {counts}") + + if not counts: + return [], [] + + sorted_vals = sorted(counts.values()) + median = sorted_vals[len(sorted_vals) // 2] + target = max(median, min_per_class) + print(f" Balancing to {target} per class") + + rng = random.Random(seed) + balanced: list[tuple[str, str]] = [] + for lbl in label_order: + items = by_class.get(lbl, []) + if not items: + print(f" WARNING: no examples for '{lbl}'", file=sys.stderr) + continue + rng.shuffle(items) + chosen = items[:target] + while len(chosen) < target: + chosen += items + balanced.extend((lbl, t) for t in chosen[:target]) + + rng.shuffle(balanced) + n_eval = max(1, int(len(balanced) * eval_frac)) + return balanced[n_eval:], balanced[:n_eval] + + +def write_tsv(path: Path, rows: list[tuple[str, str]], label_index: dict[str, int]) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + with open(path, "w") as f: + for lbl, text in rows: + idx = label_index[lbl] + f.write(f"{idx}\t0\t{text}\n") + print(f" Wrote {len(rows)} rows → {path}") + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--whetstone-bin", default="editor/build-native/whetstone_mcp") + ap.add_argument("--workspace", default=".") + ap.add_argument("--dataset-dir", default="datasets/project_benchmarks") + ap.add_argument("--out-dir", default="specialists/data/generated") + ap.add_argument("--eval-frac", type=float, default=0.15) + ap.add_argument("--seed", type=int, default=42) + ap.add_argument("--dry-run", action="store_true", + help="Print extracted examples without writing files") + args = ap.parse_args() + + binary = str(Path(args.workspace) / args.whetstone_bin) + workspace = str(Path(args.workspace).resolve()) + out_dir = Path(args.out_dir) + + projects = load_projects(Path(args.workspace) / args.dataset_dir) + + prereq_examples: list[tuple[str, str]] = [] + worker_examples: list[tuple[str, str]] = [] + verif_examples: list[tuple[str, str]] = [] + + for i, project in enumerate(projects): + print(f"[{i+1:3}/{len(projects)}] {project.get('project','?')} ({project.get('category','?')})") + spec_md = spec_markdown(project) + tasks = run_pipeline(binary, workspace, spec_md) + if not tasks: + print(" -> no tasks generated, skipping") + continue + + for task in tasks: + text = task_text(task) + if not text or len(text) < 8: + continue + + # prereq_op_selector — use text-based classifier (same signal as sprint-plan + # data) to avoid circular heuristic labels (C++ heuristic never emits + # manual-approval, so ops-based labeling produces 100% "standard"). + prereq_examples.append((text_to_prereq_class(text), text)) + + # worker_type + wt = task.get("workerType", "") + if wt in WORKER_LABEL_INDEX: + worker_examples.append((wt, text)) + + # verification_type + vt = task.get("executionContract", {}).get("verificationType", "") + if vt in VERIF_LABEL_INDEX: + verif_examples.append((vt, text)) + + print(f" -> {len(tasks)} tasks, prereq={len(prereq_examples)} " + f"worker={len(worker_examples)} verif={len(verif_examples)}") + + if args.dry_run: + print("\n=== SAMPLE EXAMPLES ===") + for name, examples in [("prereq", prereq_examples[:5]), + ("worker", worker_examples[:5]), + ("verif", verif_examples[:5])]: + print(f"\n{name}:") + for lbl, text in examples: + print(f" [{lbl}] {text[:80]}") + return + + PREREQ_ORDER = ["standard", "needs_review", "needs_approval", "full_gates"] + WORKER_ORDER = ["implementer", "reviewer", "architect", "qa"] + VERIF_ORDER = ["unit", "integration", "schema", "smoke", "docs"] + + print("\n=== prereq_op from projects ===") + train, eval_ = balance_and_split(prereq_examples, PREREQ_LABEL_INDEX, PREREQ_ORDER, + seed=args.seed, eval_frac=args.eval_frac) + write_tsv(out_dir / "prereq_op_projects_train.tsv", train, PREREQ_LABEL_INDEX) + write_tsv(out_dir / "prereq_op_projects_eval.tsv", eval_, PREREQ_LABEL_INDEX) + + print("\n=== worker_type from projects ===") + train, eval_ = balance_and_split(worker_examples, WORKER_LABEL_INDEX, WORKER_ORDER, + seed=args.seed, eval_frac=args.eval_frac) + write_tsv(out_dir / "worker_type_projects_train.tsv", train, WORKER_LABEL_INDEX) + write_tsv(out_dir / "worker_type_projects_eval.tsv", eval_, WORKER_LABEL_INDEX) + + print("\n=== verification_type from projects ===") + train, eval_ = balance_and_split(verif_examples, VERIF_LABEL_INDEX, VERIF_ORDER, + seed=args.seed, eval_frac=args.eval_frac) + write_tsv(out_dir / "verification_type_projects_train.tsv", train, VERIF_LABEL_INDEX) + write_tsv(out_dir / "verification_type_projects_eval.tsv", eval_, VERIF_LABEL_INDEX) + + print("\nDone. Combine with sprint-plan TSVs using:") + print(" cat specialists/data/generated/prereq_op_train.tsv \\") + print(" specialists/data/generated/prereq_op_projects_train.tsv \\") + print(" > specialists/data/combined/prereq_op_train.tsv") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/gen_prereq_op_data.py b/specialists/scripts/gen_prereq_op_data.py new file mode 100644 index 0000000..e8b312e --- /dev/null +++ b/specialists/scripts/gen_prereq_op_data.py @@ -0,0 +1,181 @@ +#!/usr/bin/env python3 +""" +gen_prereq_op_data.py + +Mines whetstone_DSL sprint plan files to produce labeled training data for +the prereq_op_selector specialist. + +Prerequisite op classes: + standard — [validate-intake, resolve-dependencies] + Default implementation task; no special gates needed. + needs_review — [validate-intake, architect-review] + Introduces new architecture, API surface, subsystem, or + cross-component interface. Requires architect sign-off. + needs_approval — [validate-intake, manual-approval] + Touches release, security, compliance, or production. + Requires manual gate regardless of architecture complexity. + full_gates — [validate-intake, architect-review, manual-approval] + Both architectural and production-sensitive risk present. + +Output: TSV with columns: label<TAB>hops<TAB>text + label: integer index 0–3 + 0=standard 1=needs_review 2=needs_approval 3=full_gates + hops: always 0 (unused by fabricate binary; required by format) + text: the step description text + +Usage: + python3 specialists/scripts/gen_prereq_op_data.py \\ + --sprint-dir /path/to/whetstone_DSL \\ + --train-out specialists/data/generated/prereq_op_train.tsv \\ + --eval-out specialists/data/generated/prereq_op_eval.tsv +""" +import re +import sys +import random +import argparse +from pathlib import Path +from collections import Counter + +# --------------------------------------------------------------------------- +# Classification heuristics +# --------------------------------------------------------------------------- + +# Signals that an architect needs to review this (new API, structure, design) +REVIEW_TOKENS = { + "introduce", "phase", "framework", "architecture", "design", "refactor", + "planning", "taxonomy", "grammar", "interface", "api surface", "subsystem", + "extension point", "abstraction", "decompos", "scaffold", "polyglot", + "cross-language", "cross-project", "multi-language", "orchestrat", + "migration plan", "phase plan", "protocol design", "new layer", + "foundation", "bootstrap", "initial", "phase 1", "phase 2", "phase 3", + "sprint plan", "roadmap", +} + +# Signals that a manual approval gate is required (release, security, production) +APPROVAL_TOKENS = { + "release", "deploy", "publish", "migrate", "security", "compliance", + "upgrade", "breaking change", "production", "rollout", "finalize", + "sign off", "certification", "approval", "cutover", "go-live", + "data migration", "schema migration", "breaking", "deprecat", +} + +LABEL_ORDER = ["standard", "needs_review", "needs_approval", "full_gates"] +LABEL_INDEX = {lbl: i for i, lbl in enumerate(LABEL_ORDER)} + + +def classify(text: str) -> str: + t = text.lower() + has_review = any(tok in t for tok in REVIEW_TOKENS) + has_approval = any(tok in t for tok in APPROVAL_TOKENS) + + if has_review and has_approval: + return "full_gates" + if has_review: + return "needs_review" + if has_approval: + return "needs_approval" + return "standard" + + +# --------------------------------------------------------------------------- +# Step extraction (same regex as gen_verification_type_data.py) +# --------------------------------------------------------------------------- + +STEP_RE = re.compile( + r"###\s+Step\s+\d+[a-z]?:\s+(.+?)(?:\s*\(\d+\s+tests?\))?$", + re.IGNORECASE, +) + + +def extract_steps(sprint_path: Path) -> list[tuple[str, str]]: + results = [] + for line in sprint_path.read_text(errors="replace").splitlines(): + m = STEP_RE.match(line.strip()) + if not m: + continue + desc = m.group(1).strip() + if not desc or len(desc) < 8: + continue + label = classify(desc) + results.append((label, desc)) + return results + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--sprint-dir", default=".", + help="Root of whetstone_DSL (contains sprint*_plan.md files)") + ap.add_argument("--train-out", default="specialists/data/generated/prereq_op_train.tsv") + ap.add_argument("--eval-out", default="specialists/data/generated/prereq_op_eval.tsv") + ap.add_argument("--eval-frac", type=float, default=0.15) + ap.add_argument("--seed", type=int, default=42) + args = ap.parse_args() + + sprint_dir = Path(args.sprint_dir) + plans = sorted(sprint_dir.glob("sprint*_plan.md")) + if not plans: + print(f"No sprint plan files found in {sprint_dir}", file=sys.stderr) + sys.exit(1) + print(f"Found {len(plans)} sprint plan files") + + all_examples: list[tuple[str, str]] = [] + for p in plans: + all_examples.extend(extract_steps(p)) + + counts = Counter(lbl for lbl, _ in all_examples) + print("Raw class distribution:", dict(counts)) + + # Balance: cap at 3x the smallest class, floor at 40 + min_count = min(counts.values()) + target = min(min_count * 4, sorted(counts.values())[1] if len(counts) > 1 else min_count) + target = max(target, 40) + print(f"Balancing to {target} examples per class") + + rng = random.Random(args.seed) + by_class: dict[str, list[str]] = {} + for lbl, desc in all_examples: + by_class.setdefault(lbl, []).append(desc) + + balanced: list[tuple[str, str]] = [] + for lbl in LABEL_ORDER: + descs = by_class.get(lbl, []) + if not descs: + print(f"WARNING: no examples for class '{lbl}'", file=sys.stderr) + continue + rng.shuffle(descs) + chosen = descs[:target] + while len(chosen) < target: + chosen += descs + balanced.extend((lbl, d) for d in chosen[:target]) + + rng.shuffle(balanced) + print(f"Balanced total: {len(balanced)} examples") + + n_eval = max(1, int(len(balanced) * args.eval_frac)) + eval_set = balanced[:n_eval] + train_set = balanced[n_eval:] + + Path(args.train_out).parent.mkdir(parents=True, exist_ok=True) + Path(args.eval_out).parent.mkdir(parents=True, exist_ok=True) + + def write_tsv(path, rows): + with open(path, "w") as f: + for lbl, desc in rows: + idx = LABEL_INDEX[lbl] + f.write(f"{idx}\t0\t{desc}\n") + print(f"Wrote {len(rows)} rows → {path}") + + write_tsv(args.train_out, train_set) + write_tsv(args.eval_out, eval_set) + + for split_name, split in [("train", train_set), ("eval", eval_set)]: + c = Counter(lbl for lbl, _ in split) + print(f" {split_name}: {dict(sorted(c.items()))}") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/gen_verification_type_data.py b/specialists/scripts/gen_verification_type_data.py new file mode 100644 index 0000000..860cbc4 --- /dev/null +++ b/specialists/scripts/gen_verification_type_data.py @@ -0,0 +1,204 @@ +#!/usr/bin/env python3 +""" +gen_verification_type_data.py + +Mines whetstone_DSL sprint plan files to produce labeled training data for +the verification_type specialist. + +Verification types: + unit — tests a single function/class/module in isolation + integration — tests multiple components wired together (MCP tools, APIs, pipelines) + schema — validates a data contract, struct, or schema definition + smoke — sprint integration summary / quick regression / health check + docs — documentation, packet emission, reporting artifacts + +Output: TSV with columns: label<TAB>hops<TAB>text + label: integer index 0–4 matching --labels "unit,integration,schema,smoke,docs" + 0=unit 1=integration 2=schema 3=smoke 4=docs + hops: always 0 (unused by fabricate binary; required by format) + text: the step description text + +Usage: + python3 specialists/scripts/gen_verification_type_data.py \ + --sprint-dir /path/to/whetstone_DSL \ + --train-out specialists/data/generated/verification_type_train.tsv \ + --eval-out specialists/data/generated/verification_type_eval.tsv \ + --eval-frac 0.15 +""" +import re +import sys +import random +import argparse +from pathlib import Path +from collections import Counter + +# --------------------------------------------------------------------------- +# Classification heuristics +# --------------------------------------------------------------------------- + +SCHEMA_TOKENS = { + "schema", "struct", "contract", "model", "type", "definition", + "serialization", "format", "layout", "ir", "ast", "grammar", "spec", + "signature", "interface", "protocol", "record", "field", "extension", + "packet", "payload", "shape", "descriptor", +} + +INTEGRATION_TOKENS = { + "mcp", "tool", "endpoint", "api", "wire", "pipeline", "plugin", + "register", "hook", "dispatch", "router", "connection", "rpc", + "server", "client", "session", "daemon", "handler", "bridge", + "integration", "compose", "mount", +} + +LABEL_ORDER = ["unit", "integration", "schema", "smoke", "docs"] +LABEL_INDEX = {lbl: i for i, lbl in enumerate(LABEL_ORDER)} + + +SMOKE_TOKENS = { + "integration summary", "sprint summary", "regression", "full-program", + "milestone", "health", "sanity", "smoke", "end-to-end summary", + "transition", "consolidation", "audit", "retrospective", +} + +DOCS_TOKENS = { + "documentation", "readme", "comment", "guide", "notes", + "changelog", "release notes", "publication", "publish", + "announce", "report", "artifact bundle", "evidence bundle", + "scorecard", "scoreboard", +} + +# Anything that doesn't match the above defaults to 'unit'. + + +def classify(text: str) -> str: + t = text.lower() + + # Smoke: integration summaries are almost always the last step + for tok in SMOKE_TOKENS: + if tok in t: + return "smoke" + + # Docs: documentation/publication artifacts + for tok in DOCS_TOKENS: + if tok in t: + return "docs" + + # Schema: data/contract definition + for tok in SCHEMA_TOKENS: + if tok in t: + return "schema" + + # Integration: wiring multiple components or MCP tools + for tok in INTEGRATION_TOKENS: + if tok in t: + return "integration" + + # Default: unit implementation + return "unit" + + +# --------------------------------------------------------------------------- +# Step extraction +# --------------------------------------------------------------------------- + +STEP_RE = re.compile( + r"###\s+Step\s+\d+[a-z]?:\s+(.+?)(?:\s*\(\d+\s+tests?\))?$", + re.IGNORECASE, +) + + +def extract_steps(sprint_path: Path) -> list[tuple[str, str]]: + """Return list of (label, description) from a sprint plan file.""" + results = [] + for line in sprint_path.read_text(errors="replace").splitlines(): + m = STEP_RE.match(line.strip()) + if not m: + continue + desc = m.group(1).strip() + if not desc or len(desc) < 8: + continue + label = classify(desc) + results.append((label, desc)) + return results + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--sprint-dir", default=".", + help="Root of whetstone_DSL (contains sprint*_plan.md files)") + ap.add_argument("--train-out", default="specialists/data/generated/verification_type_train.tsv") + ap.add_argument("--eval-out", default="specialists/data/generated/verification_type_eval.tsv") + ap.add_argument("--eval-frac", type=float, default=0.15) + ap.add_argument("--seed", type=int, default=42) + args = ap.parse_args() + + sprint_dir = Path(args.sprint_dir) + plans = sorted(sprint_dir.glob("sprint*_plan.md")) + if not plans: + print(f"No sprint plan files found in {sprint_dir}", file=sys.stderr) + sys.exit(1) + print(f"Found {len(plans)} sprint plan files") + + all_examples = [] + for p in plans: + for label, desc in extract_steps(p): + all_examples.append((label, desc)) + + counts = Counter(lbl for lbl, _ in all_examples) + print("Raw class distribution:", dict(counts)) + + # Balance: undersample to the second-smallest class, but cap at 3x min + min_count = min(counts.values()) + target = min(min_count * 4, sorted(counts.values())[1]) + target = max(target, 40) # floor + print(f"Balancing to {target} examples per class") + + rng = random.Random(args.seed) + by_class: dict[str, list[str]] = {} + for lbl, desc in all_examples: + by_class.setdefault(lbl, []).append(desc) + + balanced = [] + for lbl, descs in by_class.items(): + rng.shuffle(descs) + chosen = descs[:target] + if len(chosen) < target: + # oversample with repetition to reach target + while len(chosen) < target: + chosen += descs + chosen = chosen[:target] + balanced.extend((lbl, d) for d in chosen) + + rng.shuffle(balanced) + print(f"Balanced total: {len(balanced)} examples") + + # Train/eval split + n_eval = max(1, int(len(balanced) * args.eval_frac)) + eval_set = balanced[:n_eval] + train_set = balanced[n_eval:] + + Path(args.train_out).parent.mkdir(parents=True, exist_ok=True) + Path(args.eval_out).parent.mkdir(parents=True, exist_ok=True) + + def write_tsv(path, rows): + with open(path, "w") as f: + for lbl, desc in rows: + idx = LABEL_INDEX[lbl] + f.write(f"{idx}\t0\t{desc}\n") + print(f"Wrote {len(rows)} rows → {path}") + + write_tsv(args.train_out, train_set) + write_tsv(args.eval_out, eval_set) + + # Print final counts + for split_name, split in [("train", train_set), ("eval", eval_set)]: + c = Counter(lbl for lbl, _ in split) + print(f" {split_name}: {dict(sorted(c.items()))}") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/gen_worker_type_data.py b/specialists/scripts/gen_worker_type_data.py new file mode 100644 index 0000000..082e45b --- /dev/null +++ b/specialists/scripts/gen_worker_type_data.py @@ -0,0 +1,188 @@ +#!/usr/bin/env python3 +""" +gen_worker_type_data.py + +Mines whetstone_DSL sprint plan files to produce labeled training data for +the worker_type specialist. + +Worker types: + implementer — Builds, adds, creates, extends a feature or component. + Most sprint steps fall here. + reviewer — Validates, audits, checks, or analyzes existing work. + architect — Designs, plans, introduces new structural layers or frameworks. + qa — Writes or runs tests, regression checks, benchmarks. + +Output: TSV with columns: label<TAB>hops<TAB>text + label: integer index 0–3 + 0=implementer 1=reviewer 2=architect 3=qa + hops: always 0 (unused by fabricate binary; required by format) + text: the step description text + +Usage: + python3 specialists/scripts/gen_worker_type_data.py \\ + --sprint-dir /path/to/whetstone_DSL \\ + --train-out specialists/data/generated/worker_type_train.tsv \\ + --eval-out specialists/data/generated/worker_type_eval.tsv +""" +import re +import sys +import random +import argparse +from pathlib import Path +from collections import Counter + +# --------------------------------------------------------------------------- +# Classification heuristics +# --------------------------------------------------------------------------- + +# QA: writing/running tests, regression, benchmarks — check before architect +# because "integration summary" etc. are clearly QA not architecture +QA_TOKENS = { + "test", "tests", "regression", "benchmark", "benchmarks", "coverage", + "smoke", "integration summary", "sprint summary", "harness", "end-to-end", + "e2e", "golden", "fixture", "passing", "failing test", +} + +# Architect: introduces new structure, layers, frameworks, decompositions +ARCHITECT_TOKENS = { + "introduce", "phase", "architecture", "framework", "refactor", + "planning", "taxonomy", "grammar", "scaffold", "protocol", + "decompos", "organiz", "restructur", "subsystem", "foundation", + "bootstrap", "initial design", "phase plan", "design doc", + "migration plan", "sprint plan", "roadmap", "new layer", +} + +# Reviewer: validates, audits, checks existing things +REVIEWER_TOKENS = { + "review", "audit", "inspect", "diagnos", "assess", + "scan", "lint", "triage", "evaluate", "measure", +} + +# implementer is the default — everything else + +LABEL_ORDER = ["implementer", "reviewer", "architect", "qa"] +LABEL_INDEX = {lbl: i for i, lbl in enumerate(LABEL_ORDER)} + + +def classify(text: str) -> str: + t = text.lower() + + # QA first — "integration summary + regression" is clearly qa not arch + for tok in QA_TOKENS: + if tok in t: + return "qa" + + # Architect next — structural introductions + for tok in ARCHITECT_TOKENS: + if tok in t: + return "architect" + + # Reviewer — validation/audit tasks + for tok in REVIEWER_TOKENS: + if tok in t: + return "reviewer" + + return "implementer" + + +# --------------------------------------------------------------------------- +# Step extraction +# --------------------------------------------------------------------------- + +STEP_RE = re.compile( + r"###\s+Step\s+\d+[a-z]?:\s+(.+?)(?:\s*\(\d+\s+tests?\))?$", + re.IGNORECASE, +) + + +def extract_steps(sprint_path: Path) -> list[tuple[str, str]]: + results = [] + for line in sprint_path.read_text(errors="replace").splitlines(): + m = STEP_RE.match(line.strip()) + if not m: + continue + desc = m.group(1).strip() + if not desc or len(desc) < 8: + continue + label = classify(desc) + results.append((label, desc)) + return results + + +# --------------------------------------------------------------------------- +# Main +# --------------------------------------------------------------------------- + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--sprint-dir", default=".", + help="Root of whetstone_DSL (contains sprint*_plan.md files)") + ap.add_argument("--train-out", default="specialists/data/generated/worker_type_train.tsv") + ap.add_argument("--eval-out", default="specialists/data/generated/worker_type_eval.tsv") + ap.add_argument("--eval-frac", type=float, default=0.15) + ap.add_argument("--seed", type=int, default=42) + args = ap.parse_args() + + sprint_dir = Path(args.sprint_dir) + plans = sorted(sprint_dir.glob("sprint*_plan.md")) + if not plans: + print(f"No sprint plan files found in {sprint_dir}", file=sys.stderr) + sys.exit(1) + print(f"Found {len(plans)} sprint plan files") + + all_examples: list[tuple[str, str]] = [] + for p in plans: + all_examples.extend(extract_steps(p)) + + counts = Counter(lbl for lbl, _ in all_examples) + print("Raw class distribution:", dict(counts)) + + min_count = min(counts.values()) + target = min(min_count * 4, sorted(counts.values())[1] if len(counts) > 1 else min_count) + target = max(target, 40) + print(f"Balancing to {target} examples per class") + + rng = random.Random(args.seed) + by_class: dict[str, list[str]] = {} + for lbl, desc in all_examples: + by_class.setdefault(lbl, []).append(desc) + + balanced: list[tuple[str, str]] = [] + for lbl in LABEL_ORDER: + descs = by_class.get(lbl, []) + if not descs: + print(f"WARNING: no examples for class '{lbl}'", file=sys.stderr) + continue + rng.shuffle(descs) + chosen = descs[:target] + while len(chosen) < target: + chosen += descs + balanced.extend((lbl, d) for d in chosen[:target]) + + rng.shuffle(balanced) + print(f"Balanced total: {len(balanced)} examples") + + n_eval = max(1, int(len(balanced) * args.eval_frac)) + eval_set = balanced[:n_eval] + train_set = balanced[n_eval:] + + Path(args.train_out).parent.mkdir(parents=True, exist_ok=True) + Path(args.eval_out).parent.mkdir(parents=True, exist_ok=True) + + def write_tsv(path, rows): + with open(path, "w") as f: + for lbl, desc in rows: + idx = LABEL_INDEX[lbl] + f.write(f"{idx}\t0\t{desc}\n") + print(f"Wrote {len(rows)} rows → {path}") + + write_tsv(args.train_out, train_set) + write_tsv(args.eval_out, eval_set) + + for split_name, split in [("train", train_set), ("eval", eval_set)]: + c = Counter(lbl for lbl, _ in split) + print(f" {split_name}: {dict(sorted(c.items()))}") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/infer.py b/specialists/scripts/infer.py new file mode 100644 index 0000000..579dbc9 --- /dev/null +++ b/specialists/scripts/infer.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +""" +infer.py — Run a trained Whetstone specialist on text input. + +Supports both binary specialists (true/false) and N-way classifiers. + +Usage: + # Single query + python3 specialists/scripts/infer.py \ + --run-dir /mnt/storage/fabricate_runs/whetstone_verification_type \ + --labels "unit,integration,schema,smoke,docs" \ + --text "Sprint 200 integration summary + regression" + + # Batch from file (one text per line) + python3 specialists/scripts/infer.py \ + --run-dir /mnt/storage/fabricate_runs/whetstone_verification_type \ + --labels "unit,integration,schema,smoke,docs" \ + --input-file my_tasks.txt + + # Interactive REPL + python3 specialists/scripts/infer.py \ + --run-dir /mnt/storage/fabricate_runs/whetstone_verification_type \ + --labels "unit,integration,schema,smoke,docs" +""" +import sys +import argparse +import numpy as np +from pathlib import Path + +FABRICATE_TOOLS = Path("/home/bill/Documents/WhetstoneAI_Fabricate/tools") + + +def load_specialist(run_dir: str, labels_str: str | None): + """Load checkpoint, vocab, and labels. Return inference-ready state.""" + sys.path.insert(0, str(FABRICATE_TOOLS)) + from eval_with_vocab import load_checkpoint, load_vocab, load_layers, build_runtime + + ck = Path(run_dir) / "checkpoint.bin" + db = Path(run_dir) / "fabricate.db" + if not ck.exists(): + raise FileNotFoundError(f"No checkpoint.bin in {run_dir}") + if not db.exists(): + raise FileNotFoundError(f"No fabricate.db in {run_dir}") + + weights, _ = load_checkpoint(str(ck)) + _, str_to_id = load_vocab(str(db)) + layers = load_layers(str(db)) + + V = len(str_to_id) + _, emb_cnt = layers['token_embedding'] + D = emb_cnt // V + _, ff1_cnt = layers['block0.ff1_weight'] + F = ff1_cnt // D + L = sum(1 for k in layers if k.startswith('block') and k.endswith('.qkv_weight')) + cfg = dict(seq_len=96, D=D, F=F, V=V, L=L) + runtime = build_runtime(weights, layers, cfg) + + label_list = None + if labels_str: + p = Path(labels_str) + if p.exists(): + label_list = [l.strip() for l in p.read_text().split(',')] + else: + label_list = [l.strip() for l in labels_str.split(',')] + + return weights, layers, cfg, runtime, str_to_id, label_list + + +def predict(text: str, weights, layers, cfg, runtime, str_to_id, label_list) -> tuple[str, float]: + """Predict label for a single text. Returns (predicted_label, confidence).""" + sys.path.insert(0, str(FABRICATE_TOOLS)) + from eval_with_vocab import tokenize_batch, forward_batch + + token_ids = tokenize_batch([text], str_to_id, cfg['seq_len']) + logits = forward_batch(token_ids, weights, layers, cfg, runtime=runtime) # (1, V) + + if label_list: + # N-way: restrict to label vocab IDs, take argmax + label_ids = np.array([str_to_id.get(l, 0) for l in label_list], dtype=np.int32) + label_logits = logits[0, label_ids] + probs = np.exp(label_logits - label_logits.max()) + probs /= probs.sum() + idx = int(np.argmax(probs)) + return label_list[idx], float(probs[idx]) + else: + # Binary true/false + true_id = str_to_id.get('true', 3) + false_id = str_to_id.get('false', 4) + true_logit = logits[0, true_id] + false_logit = logits[0, false_id] + prob_true = float(np.exp(true_logit) / (np.exp(true_logit) + np.exp(false_logit))) + if prob_true >= 0.5: + return "true", prob_true + else: + return "false", 1.0 - prob_true + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--run-dir", required=True, + help="Directory with checkpoint.bin and fabricate.db") + ap.add_argument("--labels", default=None, + help="Comma-sep label list, e.g. 'unit,integration,schema,smoke,docs'") + ap.add_argument("--text", default=None, help="Single text to classify") + ap.add_argument("--input-file", default=None, help="File with one text per line") + ap.add_argument("--eval-tsv", default=None, + help="Evaluate accuracy on a TSV file (label<TAB>hops<TAB>text)") + args = ap.parse_args() + + print(f"Loading specialist from {args.run_dir}...", flush=True) + state = load_specialist(args.run_dir, args.labels) + weights, layers, cfg, runtime, str_to_id, label_list = state + print(f"Loaded. V={cfg['V']} D={cfg['D']} L={cfg['L']} labels={label_list}") + + if args.eval_tsv: + correct = total = 0 + with open(args.eval_tsv) as f: + for line in f: + parts = line.strip().split('\t', 2) + if len(parts) < 3: + continue + true_idx = int(parts[0]) + text = parts[2] + true_label = label_list[true_idx] if label_list else str(true_idx) + pred, conf = predict(text, *state[:-1], label_list) + if pred == true_label: + correct += 1 + total += 1 + print(f"Accuracy: {correct}/{total} = {100*correct/max(1,total):.1f}%") + return + + if args.input_file: + with open(args.input_file) as f: + lines = [l.strip() for l in f if l.strip()] + for text in lines: + pred, conf = predict(text, weights, layers, cfg, runtime, str_to_id, label_list) + print(f"{pred:15s} {conf:.3f} {text}") + return + + if args.text: + pred, conf = predict(args.text, weights, layers, cfg, runtime, str_to_id, label_list) + print(f"{pred} ({conf:.3f})") + return + + # Interactive REPL + print("Enter task descriptions (Ctrl-D to quit):") + while True: + try: + text = input("> ").strip() + except (EOFError, KeyboardInterrupt): + break + if not text: + continue + pred, conf = predict(text, weights, layers, cfg, runtime, str_to_id, label_list) + print(f" → {pred} (conf={conf:.3f})") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/prereq_op_binary_split.py b/specialists/scripts/prereq_op_binary_split.py new file mode 100644 index 0000000..aa78a0d --- /dev/null +++ b/specialists/scripts/prereq_op_binary_split.py @@ -0,0 +1,319 @@ +#!/usr/bin/env python3 +""" +prereq_op_binary_split.py + +Tests whether prereq_op is a factorizable gate: + - Instead of a 4-way flat classifier, train two binary classifiers: + 1. requires_architect_review (yes/no) + 2. requires_manual_approval (yes/no) + - Combine deterministically: + no/no → standard (0) + yes/no → needs_review (1) + no/yes → needs_approval (2) + yes/yes → full_gates (3) + +Then compare: + - flat 4-way model accuracy + - combined binary model accuracy + +Also measures: false-gate rate (how often either binary fires when neither should). + +Usage: + python3 specialists/scripts/prereq_op_binary_split.py \\ + --data-dir specialists/data/combined \\ + --runs-dir /mnt/storage/fabricate_runs \\ + --results-out specialists/eval/results/prereq_op_factorize.json +""" + +import argparse +import json +import subprocess +import sys +from pathlib import Path + +ROOT = Path(__file__).parent.parent.parent +FABRICATE = Path("/home/bill/Documents/WhetstoneAI_Fabricate") +VENV_PYTHON = FABRICATE / ".venv" / "bin" / "python3" +TRAIN_SCRIPT = ROOT / "specialists" / "scripts" / "train_specialist_pt.py" +EVAL_SCRIPT = ROOT / "specialists" / "eval" / "eval_specialist_pt.py" + +# prereq_op label mapping +LABEL_NAMES = ["standard", "needs_review", "needs_approval", "full_gates"] +# Axis decomposition: +# requires_review: needs_review(1), full_gates(3) → binary label 1 +# requires_approval: needs_approval(2), full_gates(3) → binary label 1 +REVIEW_LABELS = {1, 3} # 4-way indices that mean "needs review" +APPROVAL_LABELS = {2, 3} # 4-way indices that mean "needs approval" + + +def split_to_binary(src_tsv: Path, review_out: Path, approval_out: Path): + """Write two binary TSVs from 4-way prereq_op data.""" + review_rows = [] + approval_rows = [] + with open(src_tsv) as f: + for line in f: + parts = line.strip().split("\t", 2) + if len(parts) < 3: + continue + label_idx = int(parts[0]) + text = parts[2] + review_label = 1 if label_idx in REVIEW_LABELS else 0 + approval_label = 1 if label_idx in APPROVAL_LABELS else 0 + review_rows.append(f"{review_label}\t0\t{text}") + approval_rows.append(f"{approval_label}\t0\t{text}") + + review_out.write_text("\n".join(review_rows) + "\n") + approval_out.write_text("\n".join(approval_rows) + "\n") + print(f" review TSV: {len(review_rows)} rows → {review_out}") + print(f" approval TSV: {len(approval_rows)} rows → {approval_out}") + # Report class balance + r1 = sum(1 for r in review_rows if r.startswith("1")) + a1 = sum(1 for r in approval_rows if r.startswith("1")) + print(f" review pos={r1} neg={len(review_rows)-r1}") + print(f" approval pos={a1} neg={len(approval_rows)-a1}") + + +def train_binary(run_dir: Path, train_tsv: Path, eval_tsv: Path, + labels: str, history_out: Path) -> bool: + run_dir.mkdir(parents=True, exist_ok=True) + for f in run_dir.glob("*"): + f.unlink() + history_out.unlink(missing_ok=True) + + cmd = [ + str(VENV_PYTHON), str(TRAIN_SCRIPT), + "--dataset", str(train_tsv), + "--eval-dataset", str(eval_tsv), + "--out-dir", str(run_dir), + "--labels", labels, + "--lr", "0.001", + "--weight-decay", "0.01", + "--max-steps", "10000", + "--hidden-dim", "256", + "--layers", "2", + "--heads", "4", + "--batch-size", "256", + "--grok-loss-threshold", "0.05", + "--grok-acc-jump", "10.0", + "--stop-after-grokking-blocks", "4", + "--reset", + "--history-out", str(history_out), + ] + r = subprocess.run(cmd) + return r.returncode == 0 + + +def eval_binary(run_dir: Path, eval_tsv: Path, labels: str) -> dict | None: + ck = run_dir / "checkpoint.pt" + if not ck.exists(): + return None + r = subprocess.run( + [str(VENV_PYTHON), str(EVAL_SCRIPT), + "--checkpoint", str(ck), + "--dataset", str(eval_tsv), "--labels", labels, "--quiet"], + capture_output=True, text=True, + ) + if r.returncode != 0: + return None + try: + return json.loads(r.stdout.strip().splitlines()[-1] if r.stdout.strip() else "{}") + except json.JSONDecodeError: + return None + + +def evaluate_combined(eval_tsv: Path, review_run: Path, approval_run: Path) -> dict: + """Load both PyTorch binary models, combine predictions, compare to 4-way ground truth.""" + import numpy as np + import torch + import torch.nn as nn + + # Inline model class (mirrors train_specialist_pt.py) + def _tok(text): + return text.lower().split() + + class _Model(nn.Module): + def __init__(self, cfg): + super().__init__() + D, L, H, V, seq = cfg["hidden_dim"], cfg["layers"], cfg["heads"], cfg["vocab_size"], cfg["seq_len"] + self.tok_emb = nn.Embedding(V, D, padding_idx=0) + self.pos_emb = nn.Embedding(seq, D) + enc = nn.TransformerEncoderLayer(d_model=D, nhead=H, dim_feedforward=D*4, + dropout=0.0, batch_first=True) + self.encoder = nn.TransformerEncoder(enc, num_layers=L) + self.head = nn.Linear(D, cfg["n_labels"]) + self.seq_len = seq + + def forward(self, x): + pad_mask = (x == 0) + pos = torch.arange(x.size(1), device=x.device).unsqueeze(0) + h = self.tok_emb(x) + self.pos_emb(pos) + h = self.encoder(h, src_key_padding_mask=pad_mask) + lengths = (~pad_mask).float().sum(1, keepdim=True).clamp(min=1) + h = (h * (~pad_mask).unsqueeze(-1).float()).sum(1) / lengths + return self.head(h) + + device = torch.device("cuda" if torch.cuda.is_available() else "cpu") + + def load_pt_model(run_dir): + ckpt = torch.load(run_dir / "checkpoint.pt", map_location=device) + m = _Model(ckpt["config"]).to(device) + m.load_state_dict(ckpt["model"]) + m.eval() + vocab = ckpt.get("vocab_obj") + if vocab is None: + # reconstruct minimal vocab object + class _V: + UNK = 1 + def __init__(self, d): self.str_to_id = d; self.seq_len = ckpt["config"]["seq_len"] + def encode(self, text, seq_len): + ids = [self.str_to_id.get(t, self.UNK) for t in _tok(text)] + ids = ids[:seq_len]; ids += [0] * (seq_len - len(ids)); return ids + vocab = _V(ckpt["vocab"]) + return m, vocab, ckpt["config"]["seq_len"] + + def predict(model, vocab, seq_len, texts): + preds = [] + with torch.no_grad(): + for start in range(0, len(texts), 256): + batch = texts[start:start+256] + ids = [vocab.encode(t, seq_len) for t in batch] + x = torch.tensor(ids, dtype=torch.long, device=device) + logits = model(x).cpu().numpy() + preds.append(np.argmax(logits, axis=1)) + return np.concatenate(preds) + + # Load data + labels_gt, texts = [], [] + with open(eval_tsv) as f: + for line in f: + parts = line.strip().split("\t", 2) + if len(parts) < 3: + continue + labels_gt.append(int(parts[0])) + texts.append(parts[2]) + labels_gt = np.array(labels_gt) + n = len(labels_gt) + + review_model, review_vocab, review_seq = load_pt_model(review_run) + approval_model, approval_vocab, approval_seq = load_pt_model(approval_run) + + # Binary models: label 0 = no, label 1 = yes + review_pred = predict(review_model, review_vocab, review_seq, texts) + approval_pred = predict(approval_model, approval_vocab, approval_seq, texts) + + # Combine: review×approval → 4-way index + combined_pred = np.where( + (review_pred == 0) & (approval_pred == 0), 0, # standard + np.where( + (review_pred == 1) & (approval_pred == 0), 1, # needs_review + np.where( + (review_pred == 0) & (approval_pred == 1), 2, # needs_approval + 3 # full_gates + ) + ) + ) + + correct = (combined_pred == labels_gt) + overall = float(correct.mean() * 100) + + per_class = {} + for i, name in enumerate(LABEL_NAMES): + mask = labels_gt == i + if mask.sum() == 0: + per_class[name] = {"acc": None, "n": 0} + else: + per_class[name] = {"acc": float(correct[mask].mean() * 100), "n": int(mask.sum())} + + from collections import Counter + errors = Counter() + for gt, pred in zip(labels_gt.tolist(), combined_pred.tolist()): + if gt != pred: + errors[(LABEL_NAMES[gt], LABEL_NAMES[pred])] += 1 + + return { + "method": "binary_factorized", + "overall_accuracy": round(overall, 1), + "n_eval": n, + "per_class": per_class, + "top_errors": [{"gt": k[0], "pred": k[1], "count": v} + for k, v in errors.most_common(6)], + } + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--data-dir", default="specialists/data/combined") + ap.add_argument("--runs-dir", default="/mnt/storage/fabricate_runs") + ap.add_argument("--results-out", default="specialists/eval/results/prereq_op_factorize.json") + ap.add_argument("--skip-train", action="store_true", + help="Skip training, just eval existing checkpoints") + args = ap.parse_args() + + data_dir = Path(args.data_dir) + runs_dir = Path(args.runs_dir) + results = Path(args.results_out) + results.parent.mkdir(parents=True, exist_ok=True) + + # Intermediate data paths + tmp = Path("specialists/data/generated") + tmp.mkdir(parents=True, exist_ok=True) + review_train = tmp / "prereq_review_train.tsv" + review_eval = tmp / "prereq_review_eval.tsv" + approval_train = tmp / "prereq_approval_train.tsv" + approval_eval = tmp / "prereq_approval_eval.tsv" + + # --- Split data --- + print("=== Splitting 4-way data into two binary axes ===") + split_to_binary(data_dir / "prereq_op_train.tsv", review_train, approval_train) + split_to_binary(data_dir / "prereq_op_eval.tsv", review_eval, approval_eval) + + review_run = runs_dir / "whetstone_prereq_review_binary" + approval_run = runs_dir / "whetstone_prereq_approval_binary" + review_hist = ROOT / "specialists" / "runs" / "prereq_review_history.json" + approval_hist = ROOT / "specialists" / "runs" / "prereq_approval_history.json" + + if not args.skip_train: + # --- Train binary models --- + print("\n=== Training: requires_architect_review (binary) ===") + train_binary(review_run, review_train, review_eval, "no,yes", review_hist) + + print("\n=== Training: requires_manual_approval (binary) ===") + train_binary(approval_run, approval_train, approval_eval, "no,yes", approval_hist) + + # --- Eval individual binary models --- + print("\n=== Evaluating binary models independently ===") + review_result = eval_binary(review_run, review_eval, "no,yes") + approval_result = eval_binary(approval_run, approval_eval, "no,yes") + + if review_result: + print(f" review binary: {review_result.get('overall_accuracy')}%") + if approval_result: + print(f" approval binary: {approval_result.get('overall_accuracy')}%") + + # --- Evaluate combined predictions on 4-way task --- + print("\n=== Evaluating combined binary → 4-way accuracy ===") + combined = evaluate_combined( + data_dir / "prereq_op_eval.tsv", review_run, approval_run + ) + print(f" Combined 4-way: {combined['overall_accuracy']}%") + print(" Per class:") + for name, stat in combined["per_class"].items(): + if stat["acc"] is not None: + print(f" {name:20s}: {stat['acc']:.1f}% (n={stat['n']})") + + output = { + "gate": "prereq_op", + "factorized": { + "requires_architect_review": review_result, + "requires_manual_approval": approval_result, + "combined_4way": combined, + }, + "note": "Compare combined_4way.overall_accuracy against flat 4-way model accuracy", + } + + results.write_text(json.dumps(output, indent=2)) + print(f"\nResults → {results}") + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/sweep_all_gates.sh b/specialists/scripts/sweep_all_gates.sh new file mode 100755 index 0000000..19c4d9d --- /dev/null +++ b/specialists/scripts/sweep_all_gates.sh @@ -0,0 +1,129 @@ +#!/usr/bin/env bash +# sweep_all_gates.sh +# +# Run capacity sweeps for all 4 specialists + prereq_op factorizability test. +# Runs sweeps sequentially (one GPU job at a time). +# +# Usage: +# bash specialists/scripts/sweep_all_gates.sh [--pilot-only] [--gates worker_type,prereq_op] +# +# Defaults: full sweep for all gates. +# +# Output: +# specialists/eval/results/<gate>_sweep.json — per-gate sweep table +# specialists/eval/results/prereq_op_factorize.json — factorize comparison +# specialists/eval/results/summary.json — all gates combined + +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +PYTHON="/home/bill/Documents/WhetstoneAI_Fabricate/.venv/bin/python3" +RUNS_DIR="/mnt/storage/fabricate_runs" +RESULTS_DIR="$ROOT/specialists/eval/results" +mkdir -p "$RESULTS_DIR" + +PILOT_ONLY="" +GATES="verification_type,worker_type,prereq_op,automatability" + +for arg in "$@"; do + case "$arg" in + --pilot-only) PILOT_ONLY="--pilot-only" ;; + --gates=*) GATES="${arg#--gates=}" ;; + esac +done + +echo "=== Whetstone Specialist Capacity Sweep ===" +echo " Gates: $GATES" +echo " Mode: ${PILOT_ONLY:-(full sweep)}" +echo " Results: $RESULTS_DIR" +echo "" + +run_sweep() { + local name="$1"; local train="$2"; local eval="$3"; local labels="$4" + if echo "$GATES" | grep -qw "$name"; then + echo "" + echo ">>> Gate: $name" + "$PYTHON" "$ROOT/specialists/scripts/capacity_sweep.py" \ + --name "$name" \ + --train "$train" \ + --eval "$eval" \ + --labels "$labels" \ + --runs-dir "$RUNS_DIR" \ + --results-out "$RESULTS_DIR/${name}_sweep.json" \ + $PILOT_ONLY \ + 2>&1 | tee "$RESULTS_DIR/${name}_sweep.log" + echo "" + fi +} + +run_sweep "verification_type" \ + "$ROOT/specialists/data/generated/verification_type_train.tsv" \ + "$ROOT/specialists/data/generated/verification_type_eval.tsv" \ + "unit,integration,schema,smoke,docs" + +run_sweep "worker_type" \ + "$ROOT/specialists/data/generated/worker_type_train.tsv" \ + "$ROOT/specialists/data/generated/worker_type_eval.tsv" \ + "implementer,reviewer,architect,qa" + +run_sweep "prereq_op" \ + "$ROOT/specialists/data/combined/prereq_op_train.tsv" \ + "$ROOT/specialists/data/combined/prereq_op_eval.tsv" \ + "standard,needs_review,needs_approval,full_gates" + +run_sweep "automatability" \ + "$ROOT/specialists/data/generated/automatability_train.tsv" \ + "$ROOT/specialists/data/generated/automatability_eval.tsv" \ + "deterministic,template,specialist,slm,llm,human" + +# prereq_op factorizability test (runs alongside the sweep) +if echo "$GATES" | grep -qw "prereq_op"; then + echo "" + echo ">>> prereq_op: factorizability test (two binary classifiers)" + "$PYTHON" "$ROOT/specialists/scripts/prereq_op_binary_split.py" \ + --data-dir "$ROOT/specialists/data/combined" \ + --runs-dir "$RUNS_DIR" \ + --results-out "$RESULTS_DIR/prereq_op_factorize.json" \ + 2>&1 | tee "$RESULTS_DIR/prereq_op_factorize.log" +fi + +# --- Combine into summary --- +"$PYTHON" - <<'PYEOF' +import json, glob, sys +from pathlib import Path + +results_dir = Path("specialists/eval/results") +summary = {"gates": {}} + +for path in sorted(results_dir.glob("*_sweep.json")): + gate = path.stem.replace("_sweep", "") + data = json.loads(path.read_text()) + best_tier = max( + (t for t in data["tiers"] if t.get("eval_accuracy")), + key=lambda t: t.get("eval_accuracy", 0), + default=None, + ) + summary["gates"][gate] = { + "best_tier": best_tier["tier"] if best_tier else None, + "best_accuracy": best_tier["eval_accuracy"] if best_tier else None, + "verdict": best_tier.get("verdict") if best_tier else None, + "guardrail_threshold": best_tier.get("guardrail_threshold") if best_tier else None, + } + +# Add factorize comparison if available +factorize_path = results_dir / "prereq_op_factorize.json" +if factorize_path.exists(): + data = json.loads(factorize_path.read_text()) + summary["gates"]["prereq_op"]["factorized_accuracy"] = ( + data.get("factorized", {}).get("combined_4way", {}).get("overall_accuracy") + ) + +out = results_dir / "summary.json" +out.write_text(json.dumps(summary, indent=2)) +print(f"\nSummary → {out}") +print(json.dumps(summary, indent=2)) +PYEOF + +echo "" +echo "=== Sweep complete ===" +echo " Full results in: $RESULTS_DIR" diff --git a/specialists/scripts/train_all_large.sh b/specialists/scripts/train_all_large.sh new file mode 100755 index 0000000..df1bd46 --- /dev/null +++ b/specialists/scripts/train_all_large.sh @@ -0,0 +1,91 @@ +#!/usr/bin/env bash +# Train all 4 specialists with a larger model (hidden_dim=512, layers=2, heads=4). +# ~10-12x the capacity of the default (hidden_dim=128, layers=1, heads=1). +# Uses combined sprint-plan + project data for prereq_op. +# +# Usage: bash specialists/scripts/train_all_large.sh +# (run from whetstone_DSL root) + +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +FABRICATE_ROOT="/home/bill/Documents/WhetstoneAI_Fabricate" +RUN_BASE="/mnt/storage/fabricate_runs" + +# Model size — ~16x the default 128/1/1 +# Note: gpu_transformer engine requires heads=1 +HIDDEN=512 +LAYERS=4 +HEADS=1 +MAX_STEPS=20000 + +mkdir -p "$ROOT/specialists/runs" + +cd "$FABRICATE_ROOT" + +run_specialist() { + local name="$1" + local train="$2" + local eval="$3" + local labels="$4" + local out="$RUN_BASE/${name}_large" + local history="$ROOT/specialists/runs/${name}_large_history.json" + local log="$ROOT/specialists/runs/${name}_large.log" + + mkdir -p "$out" + echo "=== $name (large) ===" | tee "$log" + echo " train: $train" | tee -a "$log" + echo " eval: $eval" | tee -a "$log" + echo " model: hidden=$HIDDEN layers=$LAYERS heads=$HEADS" | tee -a "$log" + echo "" | tee -a "$log" + + .venv/bin/python3 run_grokking_until.py \ + --dataset "$train" \ + --eval-dataset "$eval" \ + --out-dir "$out" \ + --labels "$labels" \ + --lr 0.0005 \ + --weight-decay 0.01 \ + --max-steps "$MAX_STEPS" \ + --hidden-dim "$HIDDEN" \ + --layers "$LAYERS" \ + --heads "$HEADS" \ + --batch-size 256 \ + --grok-loss-threshold 0.05 \ + --grok-acc-jump 10.0 \ + --stop-after-grokking-blocks 5 \ + --morph-interval 99999 \ + --reset \ + --history-out "$history" \ + 2>&1 | tee -a "$log" + + echo "" | tee -a "$log" + echo "Done: $out/checkpoint.bin" | tee -a "$log" +} + +# Run sequentially — one model at a time gets full GPU, avoids memory contention. +# prereq_op — combined sprint-plan + project data (text-labeled) +run_specialist "whetstone_prereq_op" \ + "$ROOT/specialists/data/combined/prereq_op_train.tsv" \ + "$ROOT/specialists/data/combined/prereq_op_eval.tsv" \ + "standard,needs_review,needs_approval,full_gates" + +# worker_type +run_specialist "whetstone_worker_type" \ + "$ROOT/specialists/data/generated/worker_type_train.tsv" \ + "$ROOT/specialists/data/generated/worker_type_eval.tsv" \ + "implementer,reviewer,architect,qa" + +# verification_type +run_specialist "whetstone_verification_type" \ + "$ROOT/specialists/data/generated/verification_type_train.tsv" \ + "$ROOT/specialists/data/generated/verification_type_eval.tsv" \ + "unit,integration,schema,smoke,docs" + +# automatability +run_specialist "whetstone_automatability" \ + "$ROOT/specialists/data/generated/automatability_train.tsv" \ + "$ROOT/specialists/data/generated/automatability_eval.tsv" \ + "deterministic,template,specialist,slm,llm,human" +echo "" +echo "All large-model training runs complete." diff --git a/specialists/scripts/train_automatability.sh b/specialists/scripts/train_automatability.sh new file mode 100755 index 0000000..8cc2a0a --- /dev/null +++ b/specialists/scripts/train_automatability.sh @@ -0,0 +1,64 @@ +#!/usr/bin/env bash +# Train the automatability_strategy 6-way specialist. +# +# Classes (matching AutomatabilityAnnotation.strategy in Annotation.h): +# 0 = deterministic — pure formula/rule, zero parameters +# 1 = template — fixed-pattern code generation +# 2 = specialist — bounded probabilistic (~800KB model) +# 3 = slm — complex transformation, local model (~1-7GB) +# 4 = llm — full generative reasoning (14B+) +# 5 = human — requires manual judgment +# +# Input format: "name=ClassName" +# +# NOTE: 6-way classification with sparse tiers (human=5, specialist=8 raw +# examples). Grokking may take longer — max-steps set to 10000. +# If it doesn't grok by 10k, lower LR to 0.0005 and retry. +# +# Usage: bash specialists/scripts/train_automatability.sh +# (run from whetstone_DSL root) + +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +FABRICATE_ROOT="/home/bill/Documents/WhetstoneAI_Fabricate" + +TRAIN_DATA="$ROOT/specialists/data/generated/automatability_train.tsv" +EVAL_DATA="$ROOT/specialists/data/generated/automatability_eval.tsv" +OUT_DIR="/mnt/storage/fabricate_runs/whetstone_automatability" +HISTORY="$ROOT/specialists/runs/automatability_history.json" +RUN_LOG="$ROOT/specialists/runs/automatability_training.log" + +mkdir -p "$ROOT/specialists/runs" "$OUT_DIR" + +if [[ ! -f "$TRAIN_DATA" ]]; then + echo "Missing training data. Run:" >&2 + echo " python3 specialists/scripts/gen_automatability_data.py" >&2 + exit 1 +fi + +echo "Training automatability_strategy specialist..." +echo " train: $TRAIN_DATA" +echo " eval: $EVAL_DATA" +echo " out: $OUT_DIR" +echo " log: $RUN_LOG" +echo "" + +cd "$FABRICATE_ROOT" + +.venv/bin/python3 run_grokking_until.py \ + --dataset "$TRAIN_DATA" \ + --eval-dataset "$EVAL_DATA" \ + --out-dir "$OUT_DIR" \ + --labels "deterministic,template,specialist,slm,llm,human" \ + --lr 0.001 \ + --weight-decay 0.01 \ + --max-steps 10000 \ + --grok-loss-threshold 0.10 \ + --grok-acc-jump 15.0 \ + --stop-after-grokking-blocks 3 \ + --history-out "$HISTORY" \ + 2>&1 | tee "$RUN_LOG" + +echo "" +echo "Done. Checkpoint at: $OUT_DIR/checkpoint.bin" diff --git a/specialists/scripts/train_confidence_tier.sh b/specialists/scripts/train_confidence_tier.sh new file mode 100755 index 0000000..26fd124 --- /dev/null +++ b/specialists/scripts/train_confidence_tier.sh @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +# Train the confidence_tier 3-way specialist. +# Classes: high (conf>=80) / medium (60-79) / low (<60) +# Input: "conflicts=N ambiguity=M deps=D prereqs=P queueready=yes/no" +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +FABRICATE_ROOT="/home/bill/Documents/WhetstoneAI_Fabricate" + +TRAIN_DATA="$ROOT/specialists/data/generated/confidence_tier_train.tsv" +EVAL_DATA="$ROOT/specialists/data/generated/confidence_tier_eval.tsv" +OUT_DIR="/mnt/storage/fabricate_runs/whetstone_confidence_tier" +HISTORY="$ROOT/specialists/runs/confidence_tier_history.json" +RUN_LOG="$ROOT/specialists/runs/confidence_tier_training.log" + +mkdir -p "$ROOT/specialists/runs" "$OUT_DIR" + +if [[ ! -f "$TRAIN_DATA" ]]; then + echo "Generating training data..." >&2 + python3 "$ROOT/specialists/scripts/gen_confidence_tier_data.py" \ + --train-out "$TRAIN_DATA" --eval-out "$EVAL_DATA" +fi + +cd "$FABRICATE_ROOT" + +.venv/bin/python3 run_grokking_until.py \ + --dataset "$TRAIN_DATA" \ + --eval-dataset "$EVAL_DATA" \ + --out-dir "$OUT_DIR" \ + --labels "high,medium,low" \ + --lr 0.001 \ + --weight-decay 0.01 \ + --max-steps 5000 \ + --grok-loss-threshold 0.05 \ + --grok-acc-jump 15.0 \ + --stop-after-grokking-blocks 3 \ + --history-out "$HISTORY" \ + 2>&1 | tee "$RUN_LOG" diff --git a/specialists/scripts/train_prereq_op.sh b/specialists/scripts/train_prereq_op.sh new file mode 100755 index 0000000..a8eb813 --- /dev/null +++ b/specialists/scripts/train_prereq_op.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# Train the prereq_op_selector 4-way specialist. +# +# Classes: +# 0 = standard [validate-intake, resolve-dependencies] +# 1 = needs_review [validate-intake, architect-review] +# 2 = needs_approval [validate-intake, manual-approval] +# 3 = full_gates [validate-intake, architect-review, manual-approval] +# +# Usage: bash specialists/scripts/train_prereq_op.sh +# (run from whetstone_DSL root) + +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +FABRICATE_ROOT="/home/bill/Documents/WhetstoneAI_Fabricate" + +TRAIN_DATA="$ROOT/specialists/data/generated/prereq_op_train.tsv" +EVAL_DATA="$ROOT/specialists/data/generated/prereq_op_eval.tsv" +OUT_DIR="/mnt/storage/fabricate_runs/whetstone_prereq_op" +HISTORY="$ROOT/specialists/runs/prereq_op_history.json" +RUN_LOG="$ROOT/specialists/runs/prereq_op_training.log" + +mkdir -p "$ROOT/specialists/runs" "$OUT_DIR" + +if [[ ! -f "$TRAIN_DATA" ]]; then + echo "Missing training data. Run:" >&2 + echo " python3 specialists/scripts/gen_prereq_op_data.py" >&2 + exit 1 +fi + +echo "Training prereq_op_selector specialist..." +echo " train: $TRAIN_DATA" +echo " eval: $EVAL_DATA" +echo " out: $OUT_DIR" +echo " log: $RUN_LOG" +echo "" + +cd "$FABRICATE_ROOT" + +.venv/bin/python3 run_grokking_until.py \ + --dataset "$TRAIN_DATA" \ + --eval-dataset "$EVAL_DATA" \ + --out-dir "$OUT_DIR" \ + --labels "standard,needs_review,needs_approval,full_gates" \ + --lr 0.001 \ + --weight-decay 0.01 \ + --max-steps 5000 \ + --grok-loss-threshold 0.10 \ + --grok-acc-jump 15.0 \ + --stop-after-grokking-blocks 3 \ + --history-out "$HISTORY" \ + 2>&1 | tee "$RUN_LOG" + +echo "" +echo "Done. Checkpoint at: $OUT_DIR/checkpoint.bin" diff --git a/specialists/scripts/train_specialist_pt.py b/specialists/scripts/train_specialist_pt.py new file mode 100644 index 0000000..499107c --- /dev/null +++ b/specialists/scripts/train_specialist_pt.py @@ -0,0 +1,328 @@ +#!/usr/bin/env python3 +""" +train_specialist_pt.py + +PyTorch replacement for run_grokking_until.py + whetstone_fabricate. +Trains a small transformer text classifier on TSV data. +No SQLite — saves checkpoint.pt and history.json only. + +TSV format: label_idx<TAB>0<TAB>text + +Usage: + python3 specialists/scripts/train_specialist_pt.py \ + --dataset specialists/data/generated/worker_type_train.tsv \ + --eval-dataset specialists/data/generated/worker_type_eval.tsv \ + --out-dir /tmp/runs/worker_type_small \ + --labels "implementer,reviewer,architect,qa" \ + --hidden-dim 256 --layers 2 --heads 4 \ + --lr 0.001 --weight-decay 0.01 \ + --max-steps 3000 --batch-size 256 \ + --history-out /tmp/runs/worker_type_small/history.json \ + --reset +""" + +import argparse +import json +import math +import random +import time +from pathlib import Path + +import torch +import torch.nn as nn +from torch.utils.data import DataLoader, Dataset + + +# ── Vocabulary ──────────────────────────────────────────────────────────────── + +class Vocab: + PAD, UNK = 0, 1 + + def __init__(self): + self.str_to_id = {"<pad>": 0, "<unk>": 1} + self.id_to_str = ["<pad>", "<unk>"] + + def build(self, texts): + for text in texts: + for tok in _tokenize(text): + if tok not in self.str_to_id: + self.str_to_id[tok] = len(self.id_to_str) + self.id_to_str.append(tok) + + def encode(self, text, seq_len): + ids = [self.str_to_id.get(t, self.UNK) for t in _tokenize(text)] + ids = ids[:seq_len] + ids += [self.PAD] * (seq_len - len(ids)) + return ids + + def __len__(self): + return len(self.id_to_str) + + +def _tokenize(text): + return text.lower().split() + + +# ── Dataset ─────────────────────────────────────────────────────────────────── + +class TSVDataset(Dataset): + def __init__(self, path, vocab, seq_len, label_list, build_vocab=False): + self.rows = [] + texts = [] + raw = [] + with open(path) as f: + for line in f: + parts = line.strip().split("\t", 2) + if len(parts) < 3: + continue + raw.append((int(parts[0]), parts[2])) + texts.append(parts[2]) + + if build_vocab: + vocab.build(texts) + + for label_idx, text in raw: + ids = vocab.encode(text, seq_len) + self.rows.append((torch.tensor(ids, dtype=torch.long), label_idx)) + + def __len__(self): + return len(self.rows) + + def __getitem__(self, i): + return self.rows[i] + + +# ── Model ───────────────────────────────────────────────────────────────────── + +class SpecialistModel(nn.Module): + def __init__(self, vocab_size, hidden_dim, layers, heads, n_labels, seq_len): + super().__init__() + self.seq_len = seq_len + self.tok_emb = nn.Embedding(vocab_size, hidden_dim, padding_idx=0) + self.pos_emb = nn.Embedding(seq_len, hidden_dim) + encoder_layer = nn.TransformerEncoderLayer( + d_model=hidden_dim, + nhead=heads, + dim_feedforward=hidden_dim * 4, + dropout=0.0, + batch_first=True, + ) + self.encoder = nn.TransformerEncoder(encoder_layer, num_layers=layers) + self.head = nn.Linear(hidden_dim, n_labels) + self._init_weights() + + def _init_weights(self): + for p in self.parameters(): + if p.dim() > 1: + nn.init.xavier_uniform_(p) + + def forward(self, x): + # x: (B, seq_len) + pad_mask = (x == 0) # True = ignore + pos = torch.arange(x.size(1), device=x.device).unsqueeze(0) + h = self.tok_emb(x) + self.pos_emb(pos) + h = self.encoder(h, src_key_padding_mask=pad_mask) + # Mean-pool over non-padding positions + lengths = (~pad_mask).float().sum(dim=1, keepdim=True).clamp(min=1) + h = (h * (~pad_mask).unsqueeze(-1).float()).sum(dim=1) / lengths + return self.head(h) + + +# ── Training ────────────────────────────────────────────────────────────────── + +BLOCK = 1000 # steps per eval block (matches Fabricate convention) + + +def evaluate(model, loader, device): + model.eval() + correct = total = 0 + loss_sum = 0.0 + criterion = nn.CrossEntropyLoss() + with torch.no_grad(): + for x, y in loader: + x, y = x.to(device), y.to(device) + logits = model(x) + loss_sum += criterion(logits, y).item() * len(y) + correct += (logits.argmax(1) == y).sum().item() + total += len(y) + return correct / total * 100.0, loss_sum / max(total, 1) + + +def train(args): + out_dir = Path(args.out_dir) + out_dir.mkdir(parents=True, exist_ok=True) + + ckpt_path = out_dir / "checkpoint.pt" + history_path = Path(args.history_out) if args.history_out else out_dir / "history.json" + + if args.reset: + for f in [ckpt_path, history_path]: + f.unlink(missing_ok=True) + + device = torch.device("cuda" if torch.cuda.is_available() else "cpu") + print(f"Device: {device}", flush=True) + + label_list = [l.strip() for l in args.labels.split(",")] + n_labels = len(label_list) + seq_len = args.seq_len + + # ── Vocab + data ── + vocab = Vocab() + train_ds = TSVDataset(args.dataset, vocab, seq_len, label_list, build_vocab=True) + eval_ds = TSVDataset(args.eval_dataset, vocab, seq_len, label_list, build_vocab=False) + + train_loader = DataLoader(train_ds, batch_size=args.batch_size, shuffle=True, + pin_memory=(device.type == "cuda"), num_workers=0) + eval_loader = DataLoader(eval_ds, batch_size=args.batch_size, shuffle=False, + pin_memory=(device.type == "cuda"), num_workers=0) + + print(f"Train: {len(train_ds)} examples Eval: {len(eval_ds)} examples", flush=True) + print(f"Vocab: {len(vocab)} Labels: {label_list}", flush=True) + + # ── Resume or fresh ── + start_step = 0 + history_entries = [] + best_acc = 0.0 + + model = SpecialistModel( + vocab_size=len(vocab), + hidden_dim=args.hidden_dim, + layers=args.layers, + heads=args.heads, + n_labels=n_labels, + seq_len=seq_len, + ).to(device) + + optimizer = torch.optim.AdamW( + model.parameters(), lr=args.lr, weight_decay=args.weight_decay + ) + criterion = nn.CrossEntropyLoss() + + if ckpt_path.exists() and not args.reset: + ckpt = torch.load(ckpt_path, map_location=device) + model.load_state_dict(ckpt["model"]) + optimizer.load_state_dict(ckpt["optimizer"]) + vocab = ckpt["vocab_obj"] + start_step = ckpt.get("step", 0) + if history_path.exists(): + history_entries = json.loads(history_path.read_text()).get("history", []) + best_acc = max((e.get("overall", 0.0) for e in history_entries), default=0.0) + print(f"Resuming from step {start_step}, best_acc={best_acc:.1f}%", flush=True) + + n_params = sum(p.numel() for p in model.parameters()) + print(f"Model: hidden={args.hidden_dim} layers={args.layers} heads={args.heads} " + f"params={n_params:,}", flush=True) + + # ── Load all training data onto GPU once (tiny datasets — avoids DataLoader overhead) ── + all_x = torch.stack([row[0] for row in train_ds]).to(device) + all_y = torch.tensor([row[1] for row in train_ds], dtype=torch.long, device=device) + n_train = len(all_x) + + # ── Grokking detection state ── + grokked = False + blocks_since_grok = 0 + prev_block_loss = None + prev_block_acc = 0.0 + + # ── Training loop ── + step = start_step + model.train() + block_start = time.perf_counter() + + while step < args.max_steps: + idx = torch.randint(0, n_train, (args.batch_size,), device=device) + x, y = all_x[idx], all_y[idx] + optimizer.zero_grad() + loss = criterion(model(x), y) + loss.backward() + nn.utils.clip_grad_norm_(model.parameters(), 1.0) + optimizer.step() + step += 1 + + # ── Block boundary ── + if step % BLOCK == 0 or step == args.max_steps: + eval_acc, eval_loss = evaluate(model, eval_loader, device) + elapsed = time.perf_counter() - block_start + + entry = { + "step": step, + "overall": round(eval_acc, 2), + "loss": round(eval_loss, 4), + "elapsed_s": round(elapsed, 1), + } + history_entries.append(entry) + history_path.write_text(json.dumps({"history": history_entries}, indent=2)) + + # Save checkpoint + torch.save({ + "model": model.state_dict(), + "optimizer": optimizer.state_dict(), + "vocab_obj": vocab, + "vocab": vocab.str_to_id, + "config": { + "hidden_dim": args.hidden_dim, + "layers": args.layers, + "heads": args.heads, + "n_labels": n_labels, + "seq_len": seq_len, + "vocab_size": len(vocab), + }, + "labels": label_list, + "step": step, + }, ckpt_path) + + if eval_acc > best_acc: + best_acc = eval_acc + + print(f" step={step:6d} acc={eval_acc:5.1f}% loss={eval_loss:.4f} " + f"best={best_acc:.1f}% {elapsed:.0f}s", flush=True) + + # Grokking detection + if not grokked: + loss_drop = (prev_block_loss is not None and + eval_loss < args.grok_loss_threshold) + acc_jump = (eval_acc - prev_block_acc) >= args.grok_acc_jump + if loss_drop or acc_jump: + grokked = True + print(f" *** GROKKING detected at step={step} " + f"({'loss_drop' if loss_drop else 'acc_jump'}) ***", flush=True) + + if grokked: + blocks_since_grok += 1 + if blocks_since_grok >= args.stop_after_grokking_blocks: + print(f" Stopping after {blocks_since_grok} post-grokking blocks.", flush=True) + break + + prev_block_loss = eval_loss + prev_block_acc = eval_acc + block_start = time.perf_counter() + model.train() + + print(f"\nDone. best_acc={best_acc:.1f}% steps={step} checkpoint={ckpt_path}", flush=True) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--dataset", required=True) + ap.add_argument("--eval-dataset", required=True) + ap.add_argument("--out-dir", required=True) + ap.add_argument("--labels", required=True) + ap.add_argument("--lr", type=float, default=0.001) + ap.add_argument("--weight-decay", type=float, default=0.01) + ap.add_argument("--max-steps", type=int, default=10000) + ap.add_argument("--hidden-dim", type=int, default=256) + ap.add_argument("--layers", type=int, default=2) + ap.add_argument("--heads", type=int, default=4) + ap.add_argument("--batch-size", type=int, default=256) + ap.add_argument("--seq-len", type=int, default=96) + ap.add_argument("--history-out", default=None) + ap.add_argument("--reset", action="store_true") + ap.add_argument("--grok-loss-threshold", type=float, default=0.05) + ap.add_argument("--grok-acc-jump", type=float, default=10.0) + ap.add_argument("--stop-after-grokking-blocks", type=int, default=4) + args = ap.parse_args() + train(args) + + +if __name__ == "__main__": + main() diff --git a/specialists/scripts/train_verification_type.sh b/specialists/scripts/train_verification_type.sh new file mode 100755 index 0000000..c528b22 --- /dev/null +++ b/specialists/scripts/train_verification_type.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +# Train the verification_type 5-way specialist. +# +# Uses the WhetstoneAI_Fabricate binary + run_grokking_until.py harness. +# The 5 classes map directly to the router architecture: +# label index 0 = unit, 1 = integration, 2 = schema, 3 = smoke, 4 = docs +# +# Usage: bash specialists/scripts/train_verification_type.sh +# (run from whetstone_DSL root) + +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +FABRICATE_ROOT="/home/bill/Documents/WhetstoneAI_Fabricate" + +TRAIN_DATA="$ROOT/specialists/data/generated/verification_type_train.tsv" +EVAL_DATA="$ROOT/specialists/data/generated/verification_type_eval.tsv" +OUT_DIR="/mnt/storage/fabricate_runs/whetstone_verification_type" +HISTORY="$ROOT/specialists/runs/verification_type_history.json" +RUN_LOG="$ROOT/specialists/runs/verification_type_training.log" + +mkdir -p "$ROOT/specialists/runs" "$OUT_DIR" + +if [[ ! -f "$TRAIN_DATA" ]]; then + echo "Missing training data. Run:" >&2 + echo " python3 specialists/scripts/gen_verification_type_data.py" >&2 + exit 1 +fi + +echo "Training verification_type specialist..." +echo " train: $TRAIN_DATA" +echo " eval: $EVAL_DATA" +echo " out: $OUT_DIR" +echo " log: $RUN_LOG" +echo "" + +cd "$FABRICATE_ROOT" + +.venv/bin/python3 run_grokking_until.py \ + --dataset "$TRAIN_DATA" \ + --eval-dataset "$EVAL_DATA" \ + --out-dir "$OUT_DIR" \ + --labels "unit,integration,schema,smoke,docs" \ + --lr 0.001 \ + --weight-decay 0.01 \ + --max-steps 5000 \ + --grok-loss-threshold 0.10 \ + --grok-acc-jump 15.0 \ + --stop-after-grokking-blocks 3 \ + --history-out "$HISTORY" \ + 2>&1 | tee "$RUN_LOG" + +echo "" +echo "Done. Checkpoint at: $OUT_DIR/checkpoint.bin" diff --git a/specialists/scripts/train_worker_type.sh b/specialists/scripts/train_worker_type.sh new file mode 100755 index 0000000..19e57e1 --- /dev/null +++ b/specialists/scripts/train_worker_type.sh @@ -0,0 +1,56 @@ +#!/usr/bin/env bash +# Train the worker_type 4-way specialist. +# +# Classes: +# 0 = implementer — builds/creates/extends features +# 1 = reviewer — validates/audits/checks existing work +# 2 = architect — designs/introduces new structure or frameworks +# 3 = qa — writes/runs tests, regression, benchmarks +# +# Usage: bash specialists/scripts/train_worker_type.sh +# (run from whetstone_DSL root) + +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +FABRICATE_ROOT="/home/bill/Documents/WhetstoneAI_Fabricate" + +TRAIN_DATA="$ROOT/specialists/data/generated/worker_type_train.tsv" +EVAL_DATA="$ROOT/specialists/data/generated/worker_type_eval.tsv" +OUT_DIR="/mnt/storage/fabricate_runs/whetstone_worker_type" +HISTORY="$ROOT/specialists/runs/worker_type_history.json" +RUN_LOG="$ROOT/specialists/runs/worker_type_training.log" + +mkdir -p "$ROOT/specialists/runs" "$OUT_DIR" + +if [[ ! -f "$TRAIN_DATA" ]]; then + echo "Missing training data. Run:" >&2 + echo " python3 specialists/scripts/gen_worker_type_data.py" >&2 + exit 1 +fi + +echo "Training worker_type specialist..." +echo " train: $TRAIN_DATA" +echo " eval: $EVAL_DATA" +echo " out: $OUT_DIR" +echo " log: $RUN_LOG" +echo "" + +cd "$FABRICATE_ROOT" + +.venv/bin/python3 run_grokking_until.py \ + --dataset "$TRAIN_DATA" \ + --eval-dataset "$EVAL_DATA" \ + --out-dir "$OUT_DIR" \ + --labels "implementer,reviewer,architect,qa" \ + --lr 0.001 \ + --weight-decay 0.01 \ + --max-steps 5000 \ + --grok-loss-threshold 0.10 \ + --grok-acc-jump 15.0 \ + --stop-after-grokking-blocks 3 \ + --history-out "$HISTORY" \ + 2>&1 | tee "$RUN_LOG" + +echo "" +echo "Done. Checkpoint at: $OUT_DIR/checkpoint.bin"