From 9b112b9675161de115ae629d4373c145c0eb9bae Mon Sep 17 00:00:00 2001 From: Bill Holcombe Date: Sun, 12 Apr 2026 13:08:55 -0600 Subject: [PATCH] WIP: save progress 2026-04-12 Co-Authored-By: Claude Sonnet 4.6 --- ROADMAP.md | 299 +++++++- ...e_dsl_execution_contract_shape_skeleton.md | 327 +++++++++ .../whetstone_dsl_gate_extraction_specs_v0.md | 456 ++++++++++++ .../whetstone_dsl_gate_inventory.md | 374 ++++++++++ .../whetstone_dsl_run_artifact_audit.md | 348 ++++++++++ .../whetstone_dsl_semantic_mapping.md | 367 ++++++++++ .../whetstone_dsl_training_data_strategy.md | 218 ++++++ docs/whetstone_ast_first_integration.md | 649 ++++++++++++++++++ semantic/rsa_semantic_bootstrap_v0.json | 490 +++++++++++++ sprints/SPRINT-002-rsa-semantic-bootstrap.md | 202 ++++++ ...3-whetstone-dsl-gate-semantic-bootstrap.md | 351 ++++++++++ 11 files changed, 4080 insertions(+), 1 deletion(-) create mode 100644 docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md create mode 100644 docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md create mode 100644 docs/case_studies/whetstone_dsl_gate_inventory.md create mode 100644 docs/case_studies/whetstone_dsl_run_artifact_audit.md create mode 100644 docs/case_studies/whetstone_dsl_semantic_mapping.md create mode 100644 docs/case_studies/whetstone_dsl_training_data_strategy.md create mode 100644 semantic/rsa_semantic_bootstrap_v0.json create mode 100644 sprints/SPRINT-002-rsa-semantic-bootstrap.md create mode 100644 sprints/SPRINT-003-whetstone-dsl-gate-semantic-bootstrap.md diff --git a/ROADMAP.md b/ROADMAP.md index c3f7f93..b6386b5 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -6,12 +6,115 @@ - Define the bounded gate schema contract. - Document the WhetstoneDSL case study and current specialist fleet. +### Phase 0A: AST-First Realignment + +Before `whetstone_RSA` grows much further as a hand-authored C++ library, realign +the project so its semantic source of truth is Whetstone-authored and C++ becomes +a projection target. + +This phase exists to prevent the current bootstrap scaffold from hardening into the +wrong long-term architecture. + +#### Intent + +- make RSA semantically AST-first +- keep C++ as runtime projection and execution layer +- preserve compatibility with the current scaffold while changing the source of truth + +#### Architectural rule + +The long-term source of truth for RSA should live in Whetstone semantic forms, not +in hand-authored C++ structs. + +The C++ code in this repository should be treated as: + +- runtime implementation +- projected contract types +- backend execution surface +- diagnosis and policy engine + +not the canonical definition of RSA semantics. + +#### 0A.1 Semantic entities to define first + +Define Whetstone-authored semantic objects for at least: + +- `GateDefinition` +- `DecisionRequest` +- `DecisionResult` +- `StructuredDecisionResult` +- `GateSuitability` +- `GateDiagnosis` +- `ProbeDefinition` +- `BackendSuitability` +- `GatePackage` + +These should carry the contract metadata already identified in the docs: + +- input structure and modality +- output topology and constraint +- policy stability +- gate shape +- confidence, abstain, and escalation policy +- trace fields +- backend family and packaging metadata + +#### 0A.2 Projection boundary + +Define a deliberate projection boundary from Whetstone semantic forms into: + +- C++ runtime contract structs +- schema artifacts +- benchmark configuration artifacts +- packaged runtime manifests + +The important architectural constraint is: + +- semantics are authored once +- runtime targets are projected from that source + +#### 0A.3 Migration rule for the current scaffold + +Do not continue expanding hand-authored C++ contract definitions unless they are +clearly temporary runtime placeholders. + +When a contract type changes, the preferred direction should become: + +- update semantic source +- regenerate or realign projected C++ representation + +rather than: + +- evolve C++ first +- treat docs and later AST work as catch-up + +#### 0A.4 Immediate AST-first deliverables + +- define the first RSA semantic model in Whetstone terms +- map current C++ runtime types to those semantic entities +- identify what current fields are semantic contract versus runtime-only detail +- define the projection rules into C++ +- update roadmap phases so future implementation assumes semantic-first authoring + +#### 0A.5 Scope constraint + +This does not require immediate deep modification of `whetstone_DSL` runtime code. + +It does require that future RSA design work stop assuming C++ is the authoring +home of the system. + +The immediate goal is architectural realignment, not premature full conversion. + ## Phase 1: Gate Inventory - Enumerate candidate gates from WhetstoneDSL. - Record output vocabulary, input shape, and downstream deterministic executor. - Split gates into deterministic, specialist, and escalation-first candidates. +Primary case-study inventory artifact: + +- `docs/case_studies/whetstone_dsl_gate_inventory.md` + ## Phase 2: Entropy Baseline - Define a first-pass gate entropy score. @@ -31,6 +134,194 @@ - Implement abstain and escalation policies. - Support pluggable inference backends. +### Phase 4A: Post-Suitability RSA Runtime Plan + +This phase begins only after a decision surface has already been judged to belong +in the `rsa` layer rather than `deterministic`, `slm`, `llm`, or `human`. + +The focus here is not broad suitability triage. The focus is: + +- how an RSA gate is represented +- how its shape is chosen +- how its backend is selected +- how it is packaged for efficient runtime use + +#### Design constraints + +- Keep the library focused on `whetstone_RSA`, not direct `whetstone_DSL` changes. +- Assume RSA semantic contracts are authored AST-first and projected into runtime + targets rather than defined primarily in handwritten C++. +- Support both bounded single-choice classification and bounded structured outputs + with slots from the beginning. +- Include `rl_policy` and bandit-style policies as contract placeholders, but do + not make them a primary implementation target until classical ML and transformer + backends are working and benchmarked. +- Treat hybrid inputs as first-class. A gate may consume text, symbolic state, and + numeric features together. +- Use case studies only to surface reusable abstractions. Do not let one case study + become the semantic definition of the library. + +#### 4A.1 Contract and packet model + +- Define a canonical `GateDefinition` that supports: + - bounded enum decisions + - bounded structured slot-filling decisions + - constraints + - confidence policy + - abstain policy + - escalation policy + - trace metadata +- Define a canonical `DecisionRequest` packet that supports: + - textual fields + - symbolic fields + - numeric features + - state and provenance metadata +- Define `DecisionResult` and `StructuredDecisionResult` so programs can consume + either plain label outputs or bounded slot outputs through argument-oriented + runtime calls rather than project-specific ad hoc interfaces. + +#### 4A.2 Gate-shape selection + +The library should not assume that all bounded problems should be represented as +one flat multiclass classifier. + +For each RSA candidate gate, support shape analysis across: + +- `binary` +- `multiclass` +- `factorized` +- `hierarchical` + +Initial implementation goals: + +- define metadata that describes candidate gate shapes +- define diagnostics that detect when a flat label space hides compositional + structure +- support shape-specific constraints so invalid state combinations are prevented + by contract rather than patched later + +#### 4A.3 Backend-selection layer + +Backend selection remains separate from RSA identity. + +The library should support a backend contract for at least: + +- `linear_model` +- `gradient_boosting` +- `random_forest` +- `bayesian` +- `transformer` +- `hybrid` +- `rl_policy` placeholder + +Initial implementation goals: + +- define a common backend interface for train, infer, calibrate, and trace +- allow external benchmark or AutoML-style workflows to produce backend + recommendations without forcing one internal training stack +- keep backend artifacts swappable as long as the gate contract is preserved + +#### 4A.4 Benchmark harness + +Build a reusable harness that compares both gate shape and backend family for the +same bounded decision contract. + +For each gate, the harness should compare: + +- flat multiclass against factorized or hierarchical alternatives +- classical ML backends against tiny transformer backends +- deterministic baselines where any part of the gate can be rule-complete + +Required outputs: + +- accepted-decision accuracy +- calibration quality +- abstain quality +- invalid-output rate +- downstream deterministic validation success +- latency +- memory footprint +- training and retraining cost + +#### 4A.5 Runtime packaging + +AutoML or scripted comparison should be offline or occasional, not on the hot path. + +The runtime should load a selected gate package that includes: + +- gate contract +- chosen gate shape +- chosen backend family +- trained artifact reference +- confidence thresholds +- abstain and escalation policy +- trace schema and version metadata + +The steady-state runtime model is: + +- benchmark and select offline +- package once +- load the chosen gate into memory +- serve repeated decisions efficiently +- re-run selection only when schema, data, policy, or hardware conditions change + +#### 4A.6 Invocation surface + +The library should expose explicit runtime calls rather than force consumers to +rebuild invocation conventions for each gate. + +Initial target surface: + +- `decide_class(request, gate_definition)` +- `decide_structured(request, gate_definition)` +- `diagnose_gate(gate_definition, evidence, benchmark_results)` +- `recommend_backend(gate_definition, benchmark_results)` +- `recommend_gate_shape(gate_definition, benchmark_results)` + +The exact function names may change, but the core idea should remain: consumers +pass structured arguments into a reusable RSA runtime surface. + +#### 4A.7 Diagnosis loop + +When a gate underperforms, the first response should not be automatic model-size +escalation. + +The runtime and benchmark surfaces should instead help answer: + +- should this gate stay flat or be factorized? +- is part of the task actually deterministic? +- is the label space unstable? +- is the backend family mismatched to the input packet? +- should the gate abstain earlier? + +#### 4A.8 First case-study role + +Use the memory-strategy decision only as the first end-to-end exercise for the +abstract mechanism. + +The case study should help validate: + +- mixed symbolic/text/numeric packet design +- shape comparison between flat and decomposed gates +- backend comparison between classical ML and tiny transformers +- packaging of both enum-style and slot-filling outputs + +It should not be allowed to overfit the library to one memory-management ontology. + +#### 4A.9 Immediate implementation order + +- define the AST-first semantic source for RSA contracts +- define projection rules from semantic entities into C++ runtime types +- extend schema types so both enum and structured-slot gates are first-class +- define canonical request/result packet types +- define gate-shape metadata and recommendation types +- define backend-family contracts, including placeholder RL/bandit support +- build the benchmark report format +- implement classical ML adapters first +- implement transformer adapter second +- add packaging and runtime loading +- add diagnosis and recommendation APIs on top of benchmark evidence + ## Phase 5: Whetstone Integration - Expose WhetstoneDSL gates through the library. @@ -46,6 +337,12 @@ ## Exit Criteria For Version 0 - At least one reusable gate schema format is stable. +- Both enum-style and bounded structured-slot gates are supported by the contract. - At least three WhetstoneDSL gates are benchmarked across size tiers. +- At least one gate has been compared across multiple gate shapes, not just + multiple model sizes. +- At least one gate has been compared across classical ML and transformer + backends under the same contract. - A baseline entropy-to-tier recommendation table exists. -- One real application can call the library instead of hard-coding model selection. +- One real application can call the library instead of hard-coding model + selection or backend choice. diff --git a/docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md b/docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md new file mode 100644 index 0000000..81f51f3 --- /dev/null +++ b/docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md @@ -0,0 +1,327 @@ +# WhetstoneDSL Execution Contract Shape Skeleton + +## Purpose + +This document defines the first semantic skeleton for the structured decision +surface currently spread across `whetstone_DSL` execution-contract heuristics. + +The intent is not to finalize training data or runtime wiring yet. + +The intent is to make the contract explicit enough that: + +- we know what the decision is +- we know which parts are sub-decisions +- we know which parts can stay deterministic +- we can later extract training data without flattening the structure incorrectly + +## Why This Needs A Skeleton First + +The current `whetstone_DSL` execution contract is assembled from heuristics in +`RegisterArchitectIntakeTools.h`. + +That logic currently bundles several distinct choices together: + +- target files +- required tools +- acceptance commands +- verification type +- step sequence +- cross-project targets +- resource locks + +If we treat that whole bundle as one opaque label, the gate becomes unusable for +RSA. + +If we flatten it into arbitrary string labels, we lose the structure and make the +training problem worse. + +So the correct intermediate step is a semantic skeleton. + +## Entity Name + +Recommended semantic entity: + +- `ExecutionContractShapeDecision` + +This is a skeleton decision, not yet a finalized RSA gate package. + +## Contract Boundary + +`ExecutionContractShapeDecision` answers: + +- what execution-shape packet should be attached to a taskitem before downstream + deterministic execution or validation + +It does **not** answer: + +- whether the task is globally good or bad +- how the task should be routed to human/LLM/SLM layers +- what final generated code should look like + +## Field Set + +### Core identity fields + +- `decision_id` +- `task_id` +- `source_case_study` +- `contract_version` + +These fields are bookkeeping only. + +### Input context fields + +These are the fields allowed to influence the decision. + +- `task_title` +- `task_intent` +- `requirement_ids` +- `requirement_kinds` +- `ambiguity_signals` +- `conflict_signals` +- `workspace_root` +- `project_root` +- `cross_project_intent` +- `known_target_hints` + +Rule: + +- do not include downstream execution outcomes here +- do not include later compile/test results here + +## Structured output fields + +### 1. `target_files` + +Meaning: + +- which files this task most directly acts on + +Type: + +- `array` + +Notes: + +- bounded structured selection +- likely future child decision: + `TargetFileSelectionDecision` + +### 2. `required_tools` + +Meaning: + +- which named tools or MCP operations are required by the contract + +Type: + +- `array` + +Examples: + +- `whetstone_generate_taskitems` +- `whetstone_validate_taskitem` + +Notes: + +- often partly deterministic from explicit references +- could remain deterministic longer than other fields + +### 3. `acceptance_commands` + +Meaning: + +- which concrete validation commands should be run for this task + +Type: + +- `array` + +Examples: + +- `cargo test` +- `pytest` +- `ctest` + +Notes: + +- bounded structured choice +- highly sensitive to target-file inference + +### 4. `verification_type` + +Meaning: + +- the dominant verification mode implied by the task + +Type: + +- enum + +Allowed values: + +- `unit` +- `schema` +- `docs` +- `smoke` + +Notes: + +- this is the clearest immediate child gate +- should remain explicit instead of being buried inside the larger contract + +### 5. `step_sequence` + +Meaning: + +- the bounded sequence of execution phases expected for the task + +Type: + +- `array` + +Examples: + +- `inspect_scope` +- `implement_changes` +- `apply_changes` +- `run_unit_verification` +- `run_smoke_verification` +- `review_doc_alignment` + +Notes: + +- structured output +- good later candidate for a bounded sequence or hierarchical gate + +### 6. `cross_project_targets` + +Meaning: + +- external project artifacts the task is explicitly allowed or expected to touch + +Type: + +- `array` + +Notes: + +- should be empty by default +- this field must not be inferred casually from vague mentions + +### 7. `resource_locks` + +Meaning: + +- coarse-grained lock names needed to avoid conflicting execution + +Type: + +- `array` + +Notes: + +- likely deterministic in early versions +- still part of the execution-contract shape because it affects runtime behavior + +## Child Decision Decomposition + +`ExecutionContractShapeDecision` should not remain monolithic forever. + +The expected decomposition is: + +- `VerificationTypeGate` +- `TargetFileSelectionDecision` +- `AcceptanceCommandSelectionDecision` +- `StepSequenceDecision` +- `CrossProjectTargetDecision` +- `ResourceLockPolicy` + +This parent skeleton exists so those child decisions have a coherent semantic +container. + +## Deterministic Baseline Split + +Not every field in the execution contract should become a specialist. + +### Likely deterministic for longer + +- explicit tool references +- explicit target file mentions +- resource locks when directly tied to file or subsystem identity + +### Stronger specialist candidates + +- `verification_type` +- target-file selection when multiple plausible files exist +- acceptance-command selection when language/build context is mixed +- step-sequence selection + +## Invalid States + +The skeleton should define invalid or suspicious combinations. + +Examples: + +- `verification_type = docs` with only code target files and no doc targets +- `acceptance_commands = ["cargo test"]` with only Python target files +- non-empty `cross_project_targets` without cross-project intent +- empty `target_files` plus non-empty language-specific acceptance commands + +These invalid states are important because they give the later gate contract a +real constraint surface instead of a bag of fields. + +## Training Extraction Guidance + +If historic JSON/taskitem runs are mined later, extraction should happen per +child decision, not by treating the full execution contract as one class label. + +Safe pattern: + +1. reconstruct task-local context +2. extract `verification_type` +3. extract `target_files` +4. extract `acceptance_commands` +5. extract `step_sequence` +6. preserve provenance for each extracted field + +Unsafe pattern: + +- one whole JSON contract becomes one giant class label + +## Minimal Machine-Readable Shape + +The eventual machine-readable skeleton should support at least: + +```text +ExecutionContractShapeDecision + input_context: + task_title + task_intent + ambiguity_signals + conflict_signals + workspace_root + project_root + cross_project_intent + known_target_hints + output: + target_files[] + required_tools[] + acceptance_commands[] + verification_type + step_sequence[] + cross_project_targets[] + resource_locks[] + constraints: + invalid_combinations[] +``` + +## Immediate Next Step + +Define equivalent skeletons for: + +- `TargetFileSelectionDecision` +- `WorkstreamGroupingDecision` + +Then update the semantic bootstrap artifact so `ExecutionContractShapeDecision` +exists as an explicit skeleton entity rather than only as prose. diff --git a/docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md b/docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md new file mode 100644 index 0000000..4de4976 --- /dev/null +++ b/docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md @@ -0,0 +1,456 @@ +# WhetstoneDSL Gate Extraction Specs v0 + +## Purpose + +This document defines first-pass extraction specs for turning surviving +`whetstone_DSL` artifacts into RSA-aligned training rows. + +These specs are gate-native, not taskitem-native. +Each spec describes: + +- source artifacts +- allowed context +- forbidden leaked context +- label shape +- provenance expectation +- rejection conditions + +The immediate goal is to support the first high-confidence extraction surfaces +identified in the run-artifact audit. + +## Source Priority + +Primary extraction corpus: + +- `CLionProjects/whetstone_DSL/logs/taskitem_runs/*/01_intake.json` +- `CLionProjects/whetstone_DSL/logs/taskitem_runs/*/02_generate_taskitems.json` +- `CLionProjects/whetstone_DSL/logs/taskitem_runs/*/04_validate_taskitem.json` + +Supporting context reservoirs: + +- `CLionProjects/whetstone_DSL/datasets/example_run_specs/*` +- `CLionProjects/whetstone_DSL/datasets/project_benchmarks/*` + +These supporting corpora may supply spec context, but not labels. + +## Global Extraction Rules + +### Row shape + +Every extracted row should contain: + +- `gate_id` +- `decision_context` +- `decision_label` +- `decision_provenance` +- `source_artifacts` +- `rejection_flags` + +### Provenance values + +Use one of: + +- `direct_explicit` +- `direct_bundled` +- `reconstructed` +- `untrusted` + +### Global rejection rules + +Reject a candidate row when: + +- the candidate label depends on downstream execution results +- the candidate label is only inferable from post-hoc failure or success +- the candidate label shape does not match the RSA gate contract +- required pre-decision context is missing +- the artifact mixes unrelated tasks without isolatable per-task context + +## Spec: `prereq_op_selector` + +### Status + +First extraction target. +This is the strongest current gate candidate in surviving taskitem runs. + +### Source artifacts + +- `01_intake.json` +- `02_generate_taskitems.json` +- optional: `04_validate_taskitem.json` + +### Source fields + +Context candidates: + +- `01_intake.json` + - normalized requirements + - parsed spec sections + - ambiguity signals + - conflict signals +- `02_generate_taskitems.json` + - `title` + - `milestoneId` + - `dependencyTaskIds` + - `confidence` + - `ambiguityCount` + - `reasons` + +Label source: + +- `02_generate_taskitems.json.tasks[*].prerequisiteOps` + +### Allowed context + +- task title +- local task intent +- dependency ids +- local ambiguity and conflict signals +- intake-level normalized requirements +- milestone grouping if already known at task generation time + +### Forbidden leaked context + +- acceptance commands +- target files +- required tools +- downstream execution results +- validation pass/fail outcome +- generated code + +### Label shape + +Do not train as a flat multiclass label. + +Use a factorized structured label: + +- `needs_validate_intake`: boolean +- `needs_architect_review`: boolean +- `needs_resolve_dependencies`: boolean + +Allowed source values: + +- `validate-intake` +- `architect-review` +- `resolve-dependencies` + +### Provenance + +Default provenance: + +- `direct_bundled` + +Reason: + +- the list exists explicitly, but was emitted as part of a larger taskitem + bundle rather than an isolated gate invocation + +### Rejection rules + +Reject row if: + +- `prerequisiteOps` contains values outside the accepted slot vocabulary +- the task cannot be isolated from sibling tasks in the artifact +- confidence or dependency signals are missing and the context becomes too thin +- the list appears to be post-processed rather than source-emitted + +## Spec: `target_file_selection` + +### Status + +High-confidence child decision under `execution_contract_shape`. + +### Source artifacts + +- `01_intake.json` +- `02_generate_taskitems.json` +- optional: run-spec markdown from `datasets/example_run_specs` + +### Source fields + +Context candidates: + +- normalized requirements +- parsed spec +- task title +- milestone id +- dependency ids +- local reasons + +Label source: + +- `02_generate_taskitems.json.tasks[*].executionContract.targetFiles` + +### Allowed context + +- task title +- local task intent +- relevant normalized requirements +- milestone grouping +- dependency ids +- optionally scenario family metadata from run-spec corpus + +### Forbidden leaked context + +- acceptance commands +- required tools +- execution specificity score +- validation diagnostics +- downstream worker assignment +- final implementation output + +### Label shape + +Structured bounded list of repository paths. + +Recommended first-pass representation: + +- `target_files`: ordered list of paths + +Potential later factorization: + +- `primary_surface` +- `secondary_surfaces` +- `script_surface_present` + +### Provenance + +Default provenance: + +- `direct_bundled` + +### Rejection rules + +Reject row if: + +- target files include unrelated repository surfaces not justified by task text +- path list is empty +- path list appears copied across all sibling tasks without local task variation +- path list depends on downstream implementation or validation feedback + +## Spec: `acceptance_command_selection` + +### Status + +Promising child decision under `execution_contract_shape`, but more leakage risk +than target-file selection. + +### Source artifacts + +- `01_intake.json` +- `02_generate_taskitems.json` +- `04_validate_taskitem.json` for audit only, not label derivation + +### Source fields + +Context candidates: + +- normalized requirements +- task title +- target files +- milestone id + +Label source: + +- `02_generate_taskitems.json.tasks[*].executionContract.acceptanceCommands` + +### Allowed context + +- task title +- local task intent +- target files +- normalized requirements that imply verification surface + +### Forbidden leaked context + +- validation outcome +- execution success or failure +- downstream generated code +- specificity score used as a target proxy + +### Label shape + +Structured bounded list of command identifiers. + +Recommended first-pass representation: + +- `acceptance_commands`: ordered list of commands + +### Provenance + +Default provenance: + +- `direct_bundled` + +### Rejection rules + +Reject row if: + +- commands are obviously copied template defaults unrelated to task surface +- commands require information only known after code edits +- commands are inconsistent with target files or implied verification surface + +## Spec: `required_tool_selection` + +### Status + +Useful, but higher risk of mixing model choices with framework templates. + +### Source artifacts + +- `01_intake.json` +- `02_generate_taskitems.json` +- `04_validate_taskitem.json` for audit only + +### Source fields + +Context candidates: + +- normalized requirements +- task title +- target files +- prerequisite ops + +Label source: + +- `02_generate_taskitems.json.tasks[*].executionContract.requiredTools` + +### Allowed context + +- task title +- local task intent +- prerequisite ops +- target files +- normalized requirements + +### Forbidden leaked context + +- validation-recommended tools as label source +- downstream execution traces +- final worker selection + +### Label shape + +Structured bounded list of tool ids. + +Recommended first-pass representation: + +- `required_tools`: ordered list of tool names + +### Provenance + +Default provenance: + +- `direct_bundled` + +### Rejection rules + +Reject row if: + +- the tool list is clearly framework boilerplate with no task-local variation +- the list includes post-hoc validator recommendations instead of generation-time + selections +- tool names cannot be mapped to a stable tool registry + +## Spec: `verification_type` + +### Status + +Not yet a direct label source. +Treat as reconstruction target only after child execution-contract fields are +stable. + +### Candidate reconstruction inputs + +- target files +- acceptance commands +- task title +- normalized requirements + +### Current label shape + +Bounded enum: + +- `unit` +- `schema` +- `docs` +- `smoke` + +### Current provenance + +- `reconstructed` + +### Blocking issue + +No audited artifact yet shows `verification_type` as a clean standalone field. + +## Spec: `worker_type` + +### Status + +Second-pass extraction target. + +### Current evidence + +- explicit source-code surface exists in: + - `editor/src/RoutingRules.h` + - `editor/src/WorkflowOrchestrator.h` + - `editor/src/TaskitemPoolPersistence.h` + - `editor/src/headless_rpc/DispatchPart6.h` + +### Current provenance + +- `reconstructed` + +### Blocking issue + +No clean serialized label found yet in the audited taskitem-generation artifacts. + +### Next search target + +Routing and orchestration run families, not generic taskitem-generation runs. + +## Spec: `cpp_raising_profile` + +### Status + +Second-pass extraction target. + +### Current evidence + +- explicit source-code surface exists in: + - `editor/src/mcp/RegisterCppRaisingTools.h` + - `editor/src/cpp_ir/CppErrorModelMapping.h` + - `editor/src/cpp_ir/CppBuildArtifactGenerator.h` + - `editor/src/cpp_ir/CppAsyncMappingStrategy.h` + - `editor/src/cpp_ir/CppTemplateRaisingPolicy.h` + +### Current label shape + +Bounded enum: + +- `safe-first` +- `perf-first` +- `interop-first` + +### Current provenance + +- `untrusted` + +### Blocking issue + +No clean serialized profile label found yet in the audited generic taskitem +corpora. + +### Next search target + +C++-raising-specific run artifacts and MCP execution traces. + +## Immediate Implementation Order + +1. `prereq_op_selector` +2. `target_file_selection` +3. `acceptance_command_selection` +4. `required_tool_selection` +5. `verification_type` reconstruction rule +6. `worker_type` artifact search +7. `cpp_raising_profile` artifact search diff --git a/docs/case_studies/whetstone_dsl_gate_inventory.md b/docs/case_studies/whetstone_dsl_gate_inventory.md new file mode 100644 index 0000000..35342bc --- /dev/null +++ b/docs/case_studies/whetstone_dsl_gate_inventory.md @@ -0,0 +1,374 @@ +# WhetstoneDSL Gate Inventory + +## Purpose + +This document grounds `whetstone_RSA` Phase 1 in the actual decision surfaces +currently present in `whetstone_DSL`. + +It is intentionally narrower than a general architecture document. + +The goal is to answer: + +- what bounded decisions already exist in the `whetstone_DSL` taskitem and + AST-codegen pipeline +- which of those are already deterministic +- which are immediate RSA case-study candidates +- which should become semantic skeletons first before any specialist training + +This inventory is based on the current code surfaces in: + +- `editor/src/TaskitemGeneratorV2.h` +- `editor/src/TaskitemConfidenceAmbiguity.h` +- `editor/src/RoutingEngine.h` +- `editor/src/mcp/RegisterArchitectIntakeTools.h` +- `editor/src/ast/ProjectionGenerator.h` +- `editor/src/CrossLanguageProjector.h` +- `editor/src/ConstrainedProjectionGate.h` +- `editor/src/mcp/RegisterCppRaisingTools.h` + +## Architectural Observation + +The strongest current RSA surface in `whetstone_DSL` is not the final +language-emission step. + +Once the AST and target language are fixed, most current code generation is +already deterministic: + +- target generator dispatch is deterministic +- AST concept dispatch is deterministic +- environment gating is deterministic +- C++ raising profile validation is deterministic + +That means the immediate RSA case-study value is higher in: + +- taskitem generation +- execution-contract shaping +- routing +- pre-codegen bounded profile decisions + +than in raw syntax emission. + +## Current Decision Surfaces + +### A. Taskitem decomposition and planning + +Observed decisions: + +- milestone grouping +- workstream grouping +- workstream title shape +- workstream intent aggregation +- uncertainty scoring + +Current implementation state: + +- largely heuristic and deterministic in `ScopeMilestoneDecomposer` +- not yet expressed as explicit RSA gate contracts + +Assessment: + +- milestone grouping is currently deterministic and should remain so until a real + ambiguity signal shows otherwise +- uncertainty scoring is currently formula-like and should remain deterministic +- title shaping is mixed: parts are deterministic formatting, but some grouping + and summary choices are latent semantic decisions + +RSA relevance: + +- weak immediate training target +- good semantic-skeleton target + +Suggested skeleton entities: + +- `WorkstreamGroupingDecision` +- `MilestoneGroupingDecision` +- `WorkstreamSummaryPolicy` + +### B. Taskitem worker-type assignment + +Observed decision: + +- `implementer | reviewer | architect | qa` + +Current implementation state: + +- keyword heuristic in `TaskitemGeneratorV2::inferWorkerType` +- file comment explicitly says this is a placeholder to be replaced by a + `worker_type` specialist + +Assessment: + +- clear bounded enum +- direct downstream deterministic meaning +- already known from specialist experiments + +RSA relevance: + +- immediate case-study gate + +Recommended gate class: + +- bounded multiclass gate + +Likely failure risks: + +- label-space mismatch +- missing context gate + +### C. Taskitem prerequisite-op assignment + +Observed decision: + +- `validate-intake` +- `architect-review` +- `resolve-dependencies` + +Current implementation state: + +- deterministic default plus uncertainty-based add-on in + `TaskitemGeneratorV2::inferPrerequisiteOps` +- comments explicitly note this is a placeholder for a future + `prereq_op_selector` specialist + +Assessment: + +- this is not purely one flat label +- it is closer to a bounded structured output or factorized multi-label gate + +RSA relevance: + +- immediate case-study gate +- especially important for factorization probes + +Recommended gate shape: + +- factorized or multi-label, not naive flat multiclass + +Likely failure risks: + +- factorizable gate +- label-space mismatch + +### D. Confidence / ambiguity / escalation + +Observed decisions: + +- confidence score `0..100` +- ambiguity count +- escalate `true|false` + +Current implementation state: + +- formulaic in `TaskitemConfidenceAmbiguity` + +Assessment: + +- this is the strongest current example of a deterministic-disguised-as-ML gate + +RSA relevance: + +- important as negative evidence +- should stay deterministic unless a later richer policy surface emerges + +Recommended disposition: + +- deterministic layer, not RSA + +### E. Routing decision + +Observed decisions: + +- worker type: + `deterministic | template | slm | llm | human` +- context width: + `local | file | project | cross-project` +- review required +- agent role: + `linter | refactor | generator` + +Current implementation state: + +- deterministic rule cascade in `RoutingEngine` +- explicit override path plus heuristic fallbacks + +Assessment: + +- routing is already modeled as a bounded decision surface +- the current engine is a rule baseline +- this is a strong later RSA case study, but not the first one + +RSA relevance: + +- important second-wave case study +- especially useful for comparing deterministic baseline versus RSA gate quality + +Recommended disposition: + +- keep current deterministic routing as baseline +- use it later as benchmark data for RSA substitution or hybrid routing + +### F. Execution-contract shaping + +Observed decisions in `RegisterArchitectIntakeTools`: + +- target file inference +- required tool inference +- acceptance-command inference +- cross-project target inference +- verification type inference: + `unit | schema | docs | smoke` +- step-id inference: + `inspect_scope`, `implement_changes`, `apply_changes`, + `run_unit_verification`, `run_smoke_verification`, etc. +- resource-lock inference + +Assessment by subtype: + +- target-file inference: + mixed, partly lexical and partly semantic +- required-tool inference: + bounded and meaningful +- verification type: + strong existing specialist case +- step-id inference: + bounded structured sequence; good future structured-output RSA case +- resource locks: + likely deterministic unless concurrency semantics become richer + +RSA relevance: + +- `verification_type` is immediate +- execution-contract shaping is a strong structured-output frontier + +Recommended split: + +- immediate RSA gates: + - `verification_type` +- semantic skeleton first: + - `ExecutionContractShapeDecision` + - `TargetFileSelectionDecision` + - `AcceptanceCommandSelectionDecision` + - `StepSequenceDecision` + +### G. AST projection / code generation + +Observed decisions: + +- target language selection +- AST node dispatch to language-specific visitor +- annotation-preserving cross-language projection +- environment constraint gating +- C++ raising profile: + `safe-first | perf-first | interop-first` + +Current implementation state: + +- almost entirely deterministic once target language and profile are provided + +Assessment: + +- generator dispatch itself is not the best current RSA surface +- the richer opportunity is in bounded pre-codegen policy selection, not in + token-level syntax generation + +RSA relevance: + +- immediate gate candidate: + `cpp_raising_profile` +- skeleton-first targets: + - `ProjectionProfileDecision` + - `ErrorModelPolicyDecision` + - `OwnershipMappingProfileDecision` + - `AsyncMappingPolicyDecision` + +## Inventory Table + +| Decision surface | Current form | Best current layer | +| --- | --- | --- | +| `worker_type` | bounded enum | RSA immediate | +| `prereq_op_selector` | bounded structured / factorized | RSA immediate | +| `verification_type` | bounded enum | RSA immediate | +| `confidence_tier` / `escalate` | formula-like | deterministic | +| milestone grouping | heuristic grouping | semantic skeleton first | +| workstream summary/title | mixed heuristic | semantic skeleton first | +| routing layer selection | bounded enum with strong rules | deterministic baseline, later RSA | +| target-file inference | mixed bounded selection | semantic skeleton first | +| acceptance-command inference | bounded selection | semantic skeleton first | +| step-sequence inference | bounded structured sequence | semantic skeleton first | +| resource locks | bounded selection | deterministic | +| generator dispatch by language | deterministic | deterministic | +| C++ raising profile | bounded enum | RSA immediate or near-immediate | + +## Recommended Implementation Order + +### 1. Immediate RSA gate cases + +Use the already-observed bounded gates first: + +- `verification_type` +- `worker_type` +- `prereq_op_selector` +- `cpp_raising_profile` + +These are the best first fit because they already have: + +- small output vocabularies +- deterministic downstream meaning +- clear failure consequences +- real code paths in `whetstone_DSL` + +### 2. Skeleton before specialist + +Before training more specialists, define explicit semantic skeletons for: + +- workstream grouping +- execution contract shaping +- target-file selection +- acceptance-command selection +- step-sequence selection + +These are not yet clean enough as training targets because the semantic contract +is still implicit in heuristics. + +### 3. Second-wave RSA cases + +After the skeleton pass: + +- routing layer selection +- routing context-width selection +- richer codegen profile selection +- backend-policy decisions in IR raising + +## Implication For Sprint 002 + +Sprint 002 should not jump straight from the current C++ RSA scaffold into model +training. + +It should first ensure the semantic model can express at least these categories: + +- bounded enum gate +- factorized / structured-output gate +- deterministic baseline attached to a candidate gate +- downstream deterministic executor contract +- failure diagnosis for factorization and schema drift + +That semantic coverage is required to represent the real `whetstone_DSL` +surfaces above. + +## Concrete Next Step + +Define Whetstone-authored semantic entities for the first four case-study gates: + +- `VerificationTypeGate` +- `WorkerTypeGate` +- `PrereqOpGate` +- `CppRaisingProfileGate` + +and define skeleton semantic entities for: + +- `ExecutionContractShapeDecision` +- `WorkstreamGroupingDecision` +- `TargetFileSelectionDecision` + +That would let `whetstone_RSA` move from generic architecture language into a +usable case-study-grounded semantic bootstrap. diff --git a/docs/case_studies/whetstone_dsl_run_artifact_audit.md b/docs/case_studies/whetstone_dsl_run_artifact_audit.md new file mode 100644 index 0000000..6318843 --- /dev/null +++ b/docs/case_studies/whetstone_dsl_run_artifact_audit.md @@ -0,0 +1,348 @@ +# WhetstoneDSL Run Artifact Audit + +## Purpose + +This note audits surviving `whetstone_DSL` run artifacts for RSA gate extraction. +The goal is not to treat every surviving corpus as direct training data. The goal +is to identify which artifacts preserve explicit decision outputs, which preserve +only surrounding context, and which are better treated as evaluation traces. + +## Audit Targets + +Primary paths inspected: + +- `CLionProjects/whetstone_DSL/logs/taskitem_runs` +- `CLionProjects/whetstone_DSL/datasets/example_run_specs` +- `CLionProjects/whetstone_DSL/datasets/project_benchmarks` + +Representative files inspected: + +- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162113/01_intake.json` +- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162113/02_generate_taskitems.json` +- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162113/02_generate_taskitems_raw.ndjson.json` +- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162113/04_validate_taskitem.json` +- `logs/taskitem_runs/project_benchmark_matrix_20260225_192610/cp_001_python/ab_stdout.log` +- `logs/taskitem_runs/ast_once_vs_direct_20260225_193037/cp_001/base_payload.json` +- `datasets/example_run_specs/index.jsonl` +- `datasets/example_run_specs/runspec_py_cli_todo_wrong_tool_selection_01.md` +- `datasets/project_benchmarks/common_projects_100.jsonl` + +## Source Classes + +### 1. Structured taskitem runs + +Best current extraction source: + +- `logs/taskitem_runs/*/01_intake.json` +- `logs/taskitem_runs/*/02_generate_taskitems.json` +- `logs/taskitem_runs/*/04_validate_taskitem.json` + +What these preserve: + +- pre-decision intake context +- generated taskitems +- explicit task confidence and escalation signals +- explicit prerequisite operations +- explicit execution-contract fields +- validation and specificity feedback + +Observed explicit fields in `02_generate_taskitems.json`: + +- `confidence` +- `escalate` +- `prerequisiteOps` +- `queueReady` +- `reasons` +- `resourceLocks` +- `executionContract.acceptanceCommands` +- `executionContract.requiredTools` +- `executionContract.targetFiles` +- `executionContract.stepIds` +- `executionContract.riskLevel` +- `executionContract.executionSpecificityScore` + +Observed intake fields in `01_intake.json`: + +- normalized requirements +- parsed spec structure +- conflict and ambiguity signals + +Observed validation fields in `04_validate_taskitem.json`: + +- gap and specificity diagnostics +- recommended tools +- execution-quality feedback + +Assessment: + +- highest value source for `verification_type` +- highest value source for `prereq_op_selector` +- highest value source for `execution_contract_shape` +- highest value source for child skeleton decisions such as: + - `target_file_selection` + - `acceptance_command_selection` + - `required_tool_selection` + +Constraint: + +- these files preserve outputs, but not necessarily independence of each field +- some fields may be jointly emitted in one response rather than chosen as + separate model calls +- extraction should therefore attach provenance and a decomposition method + +### 2. Raw NDJSON taskitem responses + +Representative path: + +- `logs/taskitem_runs/*/02_generate_taskitems_raw.ndjson.json` + +What these preserve: + +- near-raw tool response wrapper +- text form of the generated taskitem JSON +- tool/runtime metadata + +Assessment: + +- useful provenance source +- useful for checking whether `02_generate_taskitems.json` is a direct capture + or a normalized downstream rewrite +- not a separate supervision source by itself + +Constraint: + +- this still reflects bundled output from a single taskitem-generation call +- it does not by itself prove per-field independence + +### 3. Project benchmark matrix runs + +Representative path: + +- `logs/taskitem_runs/project_benchmark_matrix_20260225_192610/*/ab_stdout.log` + +Observed shape: + +- spec +- language +- strict mode +- path A / path B readiness +- failure reasons +- token accounting + +Assessment: + +- good evaluation corpus +- good source for comparing system modes and efficiency +- not a direct source for task-shaping gates such as `prereq_op_selector` or + `target_file_selection` + +Constraint: + +- these logs summarize end-to-end path outcomes +- they do not preserve the intermediate bounded decisions we want to supervise + +### 4. AST-once vs direct comparison runs + +Representative path: + +- `logs/taskitem_runs/ast_once_vs_direct_20260225_193037/*/base_payload.json` + +Observed shape: + +- AST node payload +- generated code +- compile/test/placeholder gates +- quality flags +- used symbols + +Assessment: + +- useful evaluation corpus for AST-first vs direct generation comparisons +- useful for downstream codegen quality measurement +- not a primary direct source for taskitem decision supervision + +Constraint: + +- preserves codegen result and gate outcomes +- does not preserve the earlier planning and task-shaping decisions + +### 5. Example run specs + +Representative paths: + +- `datasets/example_run_specs/index.jsonl` +- `datasets/example_run_specs/*.md` + +Observed shape: + +- scenario family and variant metadata +- objective +- target behavior +- anti-pattern tags +- acceptance criteria +- data capture tags + +Assessment: + +- strong context/spec source +- useful for reconstructing scenario intent and mistake class +- not a direct source of labels for bounded decisions + +### 6. Project benchmark inventories + +Representative path: + +- `datasets/project_benchmarks/*.jsonl` + +Observed shape: + +- project id +- category +- language +- natural-language spec + +Assessment: + +- good context reservoir +- good replay seed corpus +- not a direct supervision source + +## Extraction Suitability By Decision + +### Direct or near-direct candidates + +- `prereq_op_selector` + - candidate source: `02_generate_taskitems.json` + - source field: `prerequisiteOps` + - note: may still need decomposition into slot-based labels rather than raw + list copy + +- `execution_contract_shape` + - candidate source: `02_generate_taskitems.json` + - source field: `executionContract` + - note: should be split into child decisions before training + +- `target_file_selection` + - candidate source: `executionContract.targetFiles` + - note: must preserve local requirement context and touched-surface context + +- `acceptance_command_selection` + - candidate source: `executionContract.acceptanceCommands` + - note: likely partly heuristic in some runs; requires provenance tagging + +- `required_tool_selection` + - candidate source: `executionContract.requiredTools` + - note: likely mixed model-plus-template behavior + +### Reconstructible but not yet trusted as direct labels + +- `verification_type` + - may be reconstructible from execution-contract structure, validation shape, + and downstream required tools + - not yet observed as a single explicit standalone field in audited artifacts + +- `worker_type` + - current code surface exists in routing/orchestration layers: + - `editor/src/RoutingRules.h` + - `editor/src/WorkflowOrchestrator.h` + - `editor/src/TaskitemPoolPersistence.h` + - `editor/src/headless_rpc/DispatchPart6.h` + - not yet observed as a clean explicit field in the audited taskitem-run + artifacts used for task generation + - may exist elsewhere in routing or orchestration artifacts, or may need + reconstruction from execution contract and downstream path + +### Poor direct-fit sources for current Sprint 003 gates + +- `cpp_raising_profile` + - current code surface exists in C++-raising layers: + - `editor/src/mcp/RegisterCppRaisingTools.h` + - `editor/src/cpp_ir/CppErrorModelMapping.h` + - `editor/src/cpp_ir/CppBuildArtifactGenerator.h` + - `editor/src/cpp_ir/CppAsyncMappingStrategy.h` + - `editor/src/cpp_ir/CppTemplateRaisingPolicy.h` + - not surfaced in the audited generic taskitem corpora reviewed so far + - likely needs a different source family tied to C++ raising flows rather than + generic taskitem generation + +## Search Outcome Notes + +Additional source search confirms: + +- `workerType` is a real first-class concept in the current DSL routing stack +- `safe-first|perf-first|interop-first` is a real first-class concept in the + current C++ raising stack +- the audited taskitem-generation artifacts do not yet show those values as + clean serialized labels in the same way that `prerequisiteOps` and + `executionContract.*` are preserved + +Practical implication: + +- `prereq_op_selector` and execution-contract child decisions can move toward + extraction-spec work now +- `worker_type` and `cpp_raising_profile` need a second artifact search pass + over routing and raising-specific run families before we treat them as + extractable labels + +## Provenance Classes + +Every extracted training row should carry one provenance tag: + +- `direct_explicit` + - field exists directly in the artifact in bounded form + +- `direct_bundled` + - field exists directly, but as one member of a larger jointly emitted object + +- `reconstructed` + - label inferred from multiple artifact fields or execution traces + +- `untrusted` + - label candidate exists, but source independence or semantic stability is too + weak to use without further review + +Current default classification: + +- `prerequisiteOps`: `direct_bundled` +- `executionContract.targetFiles`: `direct_bundled` +- `executionContract.acceptanceCommands`: `direct_bundled` +- `executionContract.requiredTools`: `direct_bundled` +- `verification_type`: `reconstructed` +- `worker_type`: `reconstructed` +- `cpp_raising_profile`: `untrusted` + +## Immediate Recommendations + +1. Use `logs/taskitem_runs/*/{01_intake.json,02_generate_taskitems.json,04_validate_taskitem.json}` + as the first extraction corpus. + +2. Treat `datasets/example_run_specs` and `datasets/project_benchmarks` as + context reservoirs, not label sources. + +3. Keep project benchmark matrix and AST/direct corpora for evaluation and + counterfactual replay, not first-pass supervision. + +4. Build extraction specs per gate rather than per full taskitem JSON. + +5. Require each extraction spec to declare: + - source files + - context slice + - label shape + - provenance class + - rejection conditions + +## Next Audit Steps + +1. Search surviving routing/orchestration artifacts for explicit `worker_type` + labels. + +2. Search C++ raising artifacts for explicit `cpp_raising_profile` choices. + +3. Draft per-gate extraction specs for: + - `prereq_op_selector` + - `target_file_selection` + - `acceptance_command_selection` + - `required_tool_selection` + +4. Add rejection rules for low-trust bundled outputs before building any + training set. diff --git a/docs/case_studies/whetstone_dsl_semantic_mapping.md b/docs/case_studies/whetstone_dsl_semantic_mapping.md new file mode 100644 index 0000000..7efa2e7 --- /dev/null +++ b/docs/case_studies/whetstone_dsl_semantic_mapping.md @@ -0,0 +1,367 @@ +# WhetstoneDSL Semantic Mapping + +## Purpose + +This document maps the current `whetstone_DSL` implementation surfaces into the + semantic entities being established for `whetstone_RSA`. + +It exists to make Sprint 003 executable. + +The goal is not to restate architecture in the abstract. The goal is to answer: + +- which `whetstone_DSL` files currently embody the first RSA-relevant decisions +- which RSA semantic entities those files imply +- which parts are already deterministic baselines +- which parts should become explicit gate contracts +- which parts should remain skeletons before training + +## Mapping Rule + +Each `whetstone_DSL` surface should be classified into one of four roles: + +- semantic source candidate +- deterministic baseline +- runtime/executor consumer +- training-data source candidate + +This keeps the case study from collapsing implementation, semantics, and data +collection into one bucket. + +## Surface Map + +### 1. `editor/src/TaskitemGeneratorV2.h` + +Current responsibility: + +- generate taskitems from decomposed workstreams +- infer `workerType` +- infer `prerequisiteOps` +- mark queue readiness + +RSA semantic implications: + +- `worker_type` is a first-class `DecisionContract` plus `DecisionSurface` +- `prereq_op_selector` is a first-class `DecisionContract` plus structured + `DecisionSurface` + +Deterministic baseline value: + +- current keyword worker heuristic is the baseline policy +- current prerequisite-op defaulting behavior is the baseline policy + +Training-data value: + +- high, once paired with richer context and accepted downstream contracts +- especially useful for: + - task description -> `worker_type` + - task description -> prerequisite-op set + +Recommended semantic extraction: + +- `WorkerTypeGate` +- `PrereqOpGate` + +### 2. `editor/src/TaskitemConfidenceAmbiguity.h` + +Current responsibility: + +- compute confidence score +- count ambiguity +- derive escalation flag + +RSA semantic implications: + +- this is primarily a policy/baseline artifact, not a first training gate + +Deterministic baseline value: + +- extremely high +- should remain attached to future gate records as deterministic baseline + evidence + +Training-data value: + +- low as a primary specialist target +- high as a supervision source for: + - deterministic-baseline detection + - guardrail-policy comparison + +Recommended semantic extraction: + +- `DeterministicPolicyBaseline` +- not an initial RSA gate + +### 3. `editor/src/mcp/RegisterArchitectIntakeTools.h` + +Current responsibility: + +- intake normalization and taskitem generation orchestration +- execution-contract shaping +- verification-type inference +- target-file inference +- acceptance-command inference +- step-sequence inference +- cross-project-target inference + +RSA semantic implications: + +- `verification_type` is a first-class immediate gate +- execution contract shaping is a structured semantic frontier + +Deterministic baseline value: + +- very high for current heuristics +- should be preserved as baseline before any specialist substitution + +Training-data value: + +- very high if taskitem runs with context and final MCP choices still exist +- likely strongest surface for: + - `verification_type` + - `target_file_selection` + - `acceptance_command_selection` + - `execution_contract_shape` + +Recommended semantic extraction: + +- `VerificationTypeGate` +- `ExecutionContractShapeDecision` +- `TargetFileSelectionDecision` +- `AcceptanceCommandSelectionDecision` +- `StepSequenceDecision` + +### 4. `editor/src/ScopeMilestoneDecomposer.h` + +Current responsibility: + +- milestone grouping +- workstream grouping +- grouped summary/title generation +- uncertainty scoring + +RSA semantic implications: + +- grouping decisions are real semantic decisions +- but they are not yet clean enough as direct training targets + +Deterministic baseline value: + +- high + +Training-data value: + +- medium +- useful after the grouping contract is made explicit + +Recommended semantic extraction: + +- `MilestoneGroupingDecision` +- `WorkstreamGroupingDecision` +- `WorkstreamSummaryPolicy` + +### 5. `editor/src/RoutingEngine.h` + +Current responsibility: + +- choose worker layer +- choose context width +- set review flag +- derive agent role + +RSA semantic implications: + +- future second-wave RSA surface +- useful as a meta-layer above particular gates + +Deterministic baseline value: + +- extremely high +- keep this baseline intact for later RSA-versus-rule comparison + +Training-data value: + +- medium to high once real historical routing outcomes are assembled + +Recommended semantic extraction: + +- `LayerSuitabilityDecision` +- `RoutingContextDecision` + +Note: + +- these sit one layer above RSA identity itself, so they should not be confused + with an RSA gate contract + +### 6. `editor/src/mcp/RegisterCppRaisingTools.h` + +Current responsibility: + +- choose and validate C++ raising profile +- run deterministic C++ raising and packaging pipeline + +RSA semantic implications: + +- `cpp_raising_profile` is a clean bounded gate candidate + +Deterministic baseline value: + +- current manual/explicit profile selection is the baseline + +Training-data value: + +- medium +- strongest when paired with IR traits, risk annotations, and downstream review + outcomes + +Recommended semantic extraction: + +- `CppRaisingProfileGate` + +### 7. `editor/src/ConstrainedProjectionGate.h` + +Current responsibility: + +- block generation when environment constraints are violated + +RSA semantic implications: + +- this is executor-side deterministic gating +- it is not currently an RSA gate candidate + +Deterministic baseline value: + +- extremely high + +Training-data value: + +- useful as supervision for invalid-output prevention +- not useful as a primary specialist target + +Recommended semantic extraction: + +- `DeterministicConstraintGate` + +### 8. `editor/src/ast/ProjectionGenerator.h` + +Current responsibility: + +- dispatch AST node kinds to language-specific visitors + +RSA semantic implications: + +- almost none at the current stage +- this is deterministic target-language projection + +Deterministic baseline value: + +- total + +Training-data value: + +- not a primary RSA training surface + +Recommended semantic extraction: + +- no immediate RSA semantic entity +- leave in deterministic projection layer + +### 9. `editor/src/CrossLanguageProjector.h` + +Current responsibility: + +- clone AST into target language +- preserve/adapt annotations + +RSA semantic implications: + +- good future source for bounded policy choices around adaptation profiles +- current implementation is deterministic + +Deterministic baseline value: + +- high + +Training-data value: + +- low now +- potentially higher later if multiple adaptation profiles exist + +Recommended semantic extraction: + +- future `ProjectionProfileDecision` +- not an immediate Sprint 003 gate + +## Semantic Entity Alignment + +### Immediate gate entities + +| WhetstoneDSL surface | RSA entity | Status | +| --- | --- | --- | +| `TaskitemGeneratorV2::inferWorkerType` | `WorkerTypeGate` | bootstrap now | +| `TaskitemGeneratorV2::inferPrerequisiteOps` | `PrereqOpGate` | bootstrap now | +| `RegisterArchitectIntakeTools::inferVerificationType` | `VerificationTypeGate` | bootstrap now | +| `RegisterCppRaisingTools` profile enum | `CppRaisingProfileGate` | bootstrap now | + +### Skeleton-first entities + +| WhetstoneDSL surface | RSA entity | Status | +| --- | --- | --- | +| execution-contract heuristics | `ExecutionContractShapeDecision` | skeleton now | +| target-file inference | `TargetFileSelectionDecision` | skeleton now | +| grouping heuristics | `WorkstreamGroupingDecision` | skeleton now | +| grouped summary logic | `WorkstreamSummaryPolicy` | skeleton now | +| step-sequence inference | `StepSequenceDecision` | skeleton now | + +### Deterministic baselines to preserve + +| WhetstoneDSL surface | RSA role | +| --- | --- | +| `TaskitemConfidenceAmbiguity` | deterministic baseline / guardrail policy | +| `RoutingEngine` | deterministic meta-routing baseline | +| `ConstrainedProjectionGate` | deterministic constraint gate | +| `ProjectionGenerator` dispatch | deterministic projection executor | +| `CrossLanguageProjector` current adaptation | deterministic projection baseline | + +## What This Means For Integration + +The current integration path should be: + +1. capture gate semantics first +2. preserve deterministic baselines as explicit comparison points +3. identify data sources for each gate +4. defer runtime substitution until the gate contracts are stable + +The wrong integration path would be: + +1. wire RSA runtime into `whetstone_DSL` +2. guess what the gates mean later + +This case study is strong enough now to avoid that mistake. + +## Training Data Notes + +Potential high-value training data sources still likely exist in or around +`whetstone_DSL`, depending on what survived deletion: + +- taskitem pipeline run logs with context and final MCP choices +- specialist training data under `whetstone_DSL/specialists` +- the `100 easy` and `25 challenging` project sets already used for examples and + evaluation +- sprint plan corpus already used for `verification_type` + +Immediate implication: + +- do not block semantic bootstrap on recovering training data +- but treat recovered taskitem/MCP-choice runs as likely first-class supervision + once the gate contracts are fixed + +## Next Mapping Step + +The next document after this one should define skeleton field sets for: + +- `ExecutionContractShapeDecision` +- `TargetFileSelectionDecision` +- `WorkstreamGroupingDecision` + +Those are the structured surfaces most likely to become later RSA gates once the +semantic contract is explicit enough to benchmark cleanly. diff --git a/docs/case_studies/whetstone_dsl_training_data_strategy.md b/docs/case_studies/whetstone_dsl_training_data_strategy.md new file mode 100644 index 0000000..0015cb3 --- /dev/null +++ b/docs/case_studies/whetstone_dsl_training_data_strategy.md @@ -0,0 +1,218 @@ +# WhetstoneDSL Training Data Strategy + +## Purpose + +This document records the data-handling rule for turning existing +`whetstone_DSL` outputs into RSA training data. + +The immediate concern is that much of the existing material may be: + +- whole JSON responses +- whole taskitem packets +- multi-decision LLM outputs + +rather than clean one-decision-per-example supervision. + +That means the data cannot be used naively. + +## Core Rule + +Do not train per-gate specialists directly on full taskitem or full-response +outputs without reconstructing: + +- the exact local context for the decision +- the specific decision target being learned +- whether the label was independently chosen or entangled with other choices + +## Why This Matters + +The old pipeline was optimized for: + +- one LLM or SLM making several bounded choices together +- minimizing token use by sharing context across those choices + +The new RSA direction is different: + +- one specialist per bounded decision +- narrower context per decision +- explicit gate contracts + +So existing outputs may reflect: + +- coupled decisions +- prompt-order effects +- hidden shared rationale +- labels that were only valid because other fields in the same response were + chosen jointly + +If we ignore that, the training data will be semantically wrong even if it looks +large. + +## Safe Extraction Rule + +For each candidate gate, build examples as: + +- `decision_context` +- `decision_label` +- `decision_provenance` + +and not as: + +- full response blob -> label + +### `decision_context` + +Must include only the information that would have been available at the moment +that gate should fire. + +Examples: + +- `worker_type` + - task title + - task intent + - maybe requirement summary + - not downstream fields derived after routing + +- `verification_type` + - target files + - task title + - task intent + - not later acceptance-command decisions unless those are part of the gate by + contract + +- `prereq_op_selector` + - task description + - uncertainty and dependency signals if the intended gate contract includes them + - not later execution outcomes + +### `decision_label` + +Must match the new gate contract exactly. + +If the historic data uses a different shape than the new gate, do not force it. + +Instead: + +- transform it carefully +- or discard it +- or redefine the gate + +### `decision_provenance` + +Each extracted example should track: + +- source artifact +- source run id if available +- whether label was explicit or inferred +- whether the example came from: + - human-authored heuristic baseline + - LLM output + - SLM output + - deterministic reconstruction + +This is necessary because not all labels have equal trust. + +## Gate-Specific Warnings + +### `worker_type` + +This is likely recoverable from older taskitems, but only if the worker role was +explicitly present and chosen before downstream execution shaping. + +Risk: + +- label polluted by later execution-contract logic + +### `verification_type` + +This is relatively safe because it can often be reconstructed from target files +and acceptance intent. + +Risk: + +- historic labels may include joint decisions with acceptance-command shaping + +### `prereq_op_selector` + +This is the riskiest early gate for naive extraction. + +Reason: + +- older systems may have emitted a flat list of ops as one bundle +- the new semantic contract treats this as structured or factorized + +Rule: + +- do not flatten structured op bundles into arbitrary multiclass labels + +### `cpp_raising_profile` + +This is likely only useful if old runs actually recorded explicit profile choice +with enough IR context. + +If not, this gate may need to start with synthetic or policy-authored data rather +than mined response logs. + +## Easy/Challenging Project Corpora + +The `100 easy` and `25 challenging` project sets are valuable, but they are not +automatically specialist training sets. + +They are best treated as: + +- context reservoirs +- replay/evaluation corpora +- example generators for extracting many per-gate decisions + +The right pattern is: + +1. replay or parse the project artifact +2. isolate one decision surface at a time +3. extract only the local context relevant to that gate +4. record the corresponding bounded label + +This should produce a gate-native dataset rather than a project-native dataset. + +## Compatibility Constraint + +We should not assume the new specialists must fit perfectly into the exact old +taskitem bundle shape. + +There are two valid outcomes: + +### Option A: Fit specialists into current taskitems + +Use when: + +- the old taskitem structure already matches clean gate boundaries +- the decision context can be isolated without distortion + +### Option B: Rebuild taskitems around explicit gates + +Use when: + +- the current taskitems bundle too many coupled decisions +- the old layout hides which context belongs to which choice +- structured gates are being forced into flat labels + +This is not failure. +It may be the correct result of moving from one-model-many-decisions to +many-model-one-decision architecture. + +## Recommended Next Step + +Before mining historic runs, define extraction specs for the first four gates: + +- `verification_type` +- `worker_type` +- `prereq_op_selector` +- `cpp_raising_profile` + +Each extraction spec should state: + +- allowed input fields +- forbidden leaked fields +- label shape +- trust level of recovered labels + +Only after that should we parse old JSON outputs or taskitem runs. diff --git a/docs/whetstone_ast_first_integration.md b/docs/whetstone_ast_first_integration.md index dde0b15..7e823f8 100644 --- a/docs/whetstone_ast_first_integration.md +++ b/docs/whetstone_ast_first_integration.md @@ -221,3 +221,652 @@ It should become: That is the architecture most consistent with both Whetstone and the long-term goals of RSA. + +## Bootstrap Mapping From The Current C++ Scaffold + +The current C++ code is useful bootstrap material, but it should now be treated as +input to the semantic model rather than the long-term definition of that model. + +This section inventories the current scaffold and classifies each piece as either: + +- semantic contract that should move into Whetstone-authored form +- runtime behavior that should remain in projected C++ +- bootstrap/demo code that should not define long-term architecture + +### Current C++ Semantic Surface + +From `include/whetstone_rsa/types.h`, the current scaffold already expresses the +core semantic vocabulary that should be re-authored in Whetstone form: + +- ontology enums + - `InputStructure` + - `InputModality` + - `OutputTopology` + - `OutputConstraint` + - `LabelStability` + - `TargetLayer` +- diagnosis enums + - `FailureClass` + - `Intervention` + - `DeployMode` + - `RiskTier` +- contract entities + - `GateDefinition` + - `GateEvidence` + - `PolicyContext` + - `ProbeRequest` + - `ProbeResult` + - `GateDiagnosis` + - `GateSuitability` + - `PolicyRecommendation` + +These should not remain primarily hand-authored C++ concepts. + +They should become semantic entities in the Whetstone source of truth. + +### Recommended Whetstone Semantic Entities + +The current scaffold suggests the following first semantic model. + +#### 1. `DecisionContract` + +This should become the semantic home for the ontology-level fields that currently +sit inside `GateDefinition`. + +Proposed fields: + +- `gate_id` +- `task_family` +- `input_structure` +- `input_modality` +- `output_topology` +- `output_constraint` +- `risk_tier` +- `policy_stability` +- `deterministic_baseline_available` +- `supports_abstain` +- `metadata` + +This is the contract boundary, independent of backend family. + +#### 2. `DecisionSurface` + +The current `GateDefinition.labels` plus `candidate_factorizations` really describe +the shape of the output space, not just the gate identity. + +Proposed fields: + +- `output_labels` +- `slot_schema` +- `candidate_factorizations` +- `candidate_gate_shapes` +- `invalid_combinations` +- `deterministic_prepass_options` + +This gives the AST layer a place to describe flat multiclass, hierarchical, or +factorized designs without forcing that structure to remain implicit in C++. + +#### 3. `DecisionRequestModel` + +The current C++ scaffold does not yet have a first-class request packet type +separate from evaluation/demo inputs. + +That should be corrected in the AST-first model. + +Proposed fields: + +- `packet_id` +- `text_fields` +- `symbolic_fields` +- `numeric_fields` +- `context_fields` +- `state_fields` +- `provenance` + +This entity should support both enum-style gates and bounded structured outputs. + +#### 4. `DecisionResultModel` + +The current scaffold implicitly assumes label selection and policy behavior, but the +AST-first model should make result structure explicit. + +Proposed fields: + +- `decision_id` +- `selected_label` +- `filled_slots` +- `confidence` +- `abstain` +- `retry_strategy` +- `escalation_target` +- `trace_packet` + +This should later project into both C++ runtime structs and packaging artifacts. + +#### 5. `DiagnosisModel` + +The current `GateDiagnosis` is already close to a semantic object and should be +preserved almost directly, but moved into Whetstone-authored form. + +Proposed fields: + +- `primary_failure_class` +- `secondary_failure_classes` +- `supporting_signals` +- `recommended_probes` +- `recommended_interventions` +- `recommended_policy` +- `confidence` + +#### 6. `ProbeDefinition` + +The current `Probe` interface and `ProbeRequest` / `ProbeResult` pair indicate that +probes are part of the semantic model, not just implementation detail. + +The AST layer should explicitly represent: + +- `probe_id` +- `applies_to_gate_shapes` +- `required_evidence_fields` +- `produced_signals` +- `failure_hints` +- `intervention_hints` + +Concrete runtime probes such as `confidence_threshold_probe` should then be C++ +implementations of these declared probe contracts. + +#### 7. `SuitabilityAssessment` + +The current `GateSuitability` type should become a semantic output model for the +pre-RSA decision that determines whether the problem belongs in the RSA layer at all. + +Proposed fields: + +- `recommended` +- `target_layer` +- `rationale` +- `bounded_output_confidence` +- `deterministic_baseline_available` +- `label_stability_risk` +- `suggested_gate_shape` + +#### 8. `BackendSuitability` + +This does not yet exist in code, but the roadmap now requires it. + +Proposed fields: + +- `gate_id` +- `candidate_backends` +- `recommended_backend` +- `rationale` +- `confidence` +- `benchmark_refs` + +This must be semantic-first, not added ad hoc in C++ later. + +#### 9. `GatePackage` + +The runtime needs a durable packaged form for serving decisions after offline +selection and calibration. + +Proposed fields: + +- `gate_id` +- `contract_ref` +- `gate_shape` +- `backend_family` +- `artifact_ref` +- `thresholds` +- `abstain_policy` +- `escalation_policy` +- `trace_schema` +- `version` + +### What Should Remain Runtime-Only In C++ + +The following concepts should remain projected runtime behavior rather than become +the semantic source of truth: + +- `Probe` virtual interface +- `ProbeRegistry` +- `DiagnosisEngine` +- `SuitabilityAssessor` +- backend adapter implementations +- calibration code +- packaging loaders +- benchmark runners + +These are execution mechanisms, not the semantic model itself. + +### What In The Current Scaffold Is Only Bootstrap + +The following parts are useful but should not steer the long-term semantic design: + +- `examples/diagnose_demo.cpp` +- the current hand-authored stringification helpers +- any current field arrangement that exists only because it was convenient for the + first C++ scaffold + +### Immediate Migration Map + +The practical next move should be: + +1. Represent the current C++ contract entities above as Whetstone semantic objects. +2. Mark which current fields are semantic contract versus runtime-only detail. +3. Treat `types.h` as a temporary projected contract snapshot, not the authoring + home of the model. +4. Add projection rules so future C++ runtime types are derived from the semantic + source rather than independently evolved. + +### Most Important Boundary To Preserve + +The current scaffold already mixes two different layers that must stay separate: + +- semantic contract +- runtime execution strategy + +If that separation is preserved now, `whetstone_RSA` can become genuinely AST-first. +If it is not preserved, the project will keep drifting into a C++-first runtime +with AST plans attached afterward. + +## First RSA Semantic Model + +This section defines the first concrete semantic model that should become the +source of truth for `whetstone_RSA`. + +It is intentionally small and close to the current scaffold so migration can +start immediately without pretending the architecture is already finished. + +The design rule is: + +- semantic entities describe contract and meaning +- projected C++ types describe runtime execution + +### Model Goals + +The first semantic model must support all of the following from the beginning: + +- bounded enum decisions +- bounded structured slot-filling decisions +- suitability assessment before a gate is adopted +- diagnosis after a gate underperforms +- gate-shape comparison +- backend-selection metadata +- runtime packaging metadata + +### Semantic Entity Set + +The first semantic model should consist of these entities: + +- `DecisionContract` +- `DecisionSurface` +- `DecisionRequestModel` +- `DecisionResultModel` +- `SuitabilityAssessment` +- `DiagnosisModel` +- `ProbeDefinition` +- `BackendSuitability` +- `GatePackage` + +### 1. `DecisionContract` + +This is the semantic identity of an RSA gate. + +It answers: + +- what decision surface exists +- what kind of inputs it consumes +- what kind of bounded outputs it promises +- what policy constraints govern deployment + +Proposed fields: + +```text +DecisionContract + gate_id + task_family + input_structure + input_modality + output_topology + output_constraint + risk_tier + policy_stability + deterministic_baseline_available + supports_abstain + metadata +``` + +Notes: + +- `policy_stability` should replace C++ naming that implies only labels drift. +- this entity should not include backend-family decisions +- this entity should not include benchmark evidence + +### 2. `DecisionSurface` + +This is the semantic description of the output space. + +It is separate from `DecisionContract` because two gates can share similar +contract properties but differ in shape: + +- flat enum choice +- factorized decision +- hierarchical route +- bounded slot fill + +Proposed fields: + +```text +DecisionSurface + gate_id + primary_mode // enum | structured | hybrid + output_labels + slot_schema + candidate_gate_shapes // binary | multiclass | factorized | hierarchical + candidate_factorizations + invalid_combinations + deterministic_prepass_options + downstream_executor_kind +``` + +Notes: + +- `slot_schema` is required from version 0 even if many early gates are enum-only +- `invalid_combinations` should make structured outputs safe by contract +- `downstream_executor_kind` should record what deterministic system consumes the result + +### 3. `DecisionRequestModel` + +This is the semantic input packet model for runtime use. + +It should support hybrid packets directly instead of forcing a project to flatten +everything into one string. + +Proposed fields: + +```text +DecisionRequestModel + packet_id + gate_id + text_fields + symbolic_fields + numeric_fields + context_fields + state_fields + provenance +``` + +Expected usage: + +- `text_fields` + - natural language summaries + - extracted comments + - short requirement packets +- `symbolic_fields` + - AST node kinds + - annotations + - enums + - dependency graphs +- `numeric_fields` + - counts + - scores + - ratios + - latency or complexity features + +### 4. `DecisionResultModel` + +This is the canonical semantic output model for runtime decisions. + +Proposed fields: + +```text +DecisionResultModel + decision_id + gate_id + selected_label + filled_slots + confidence + abstain + retry_strategy + escalation_target + trace_packet +``` + +Notes: + +- enum gates mainly use `selected_label` +- structured gates mainly use `filled_slots` +- hybrid outputs may use both + +### 5. `SuitabilityAssessment` + +This is the semantic result of asking whether a problem belongs in RSA at all. + +Proposed fields: + +```text +SuitabilityAssessment + gate_id + recommended + target_layer // deterministic | rsa | slm | llm | human + rationale + bounded_output_confidence + deterministic_baseline_available + policy_stability_risk + suggested_gate_shape +``` + +This entity is intentionally separate from diagnosis and backend selection. + +### 6. `DiagnosisModel` + +This is the semantic result of evaluating a weak or risky gate. + +Proposed fields: + +```text +DiagnosisModel + gate_id + primary_failure_class + secondary_failure_classes + supporting_signals + recommended_probes + recommended_interventions + recommended_policy + confidence +``` + +This should remain close to the current C++ `GateDiagnosis`. + +### 7. `ProbeDefinition` + +This is the semantic declaration of a diagnostic probe. + +The semantic layer should define what a probe means before runtime C++ implements +how it executes. + +Proposed fields: + +```text +ProbeDefinition + probe_id + applies_to_gate_shapes + required_contract_fields + required_evidence_fields + produced_signals + failure_hints + intervention_hints +``` + +Concrete examples: + +- `confidence_threshold_probe` +- `class_confusion_probe` +- `factorization_probe` +- `deterministic_rule_probe` +- `context_width_probe` + +### 8. `BackendSuitability` + +This is the semantic result of backend comparison after a gate has already been +judged to belong in RSA. + +Proposed fields: + +```text +BackendSuitability + gate_id + candidate_backends + recommended_backend + rationale + confidence + benchmark_refs +``` + +Expected backend families: + +- `linear_model` +- `gradient_boosting` +- `random_forest` +- `bayesian` +- `transformer` +- `hybrid` +- `rl_policy` + +`rl_policy` should exist in the model now even if it remains an unimplemented +runtime placeholder for the first version. + +### 9. `GatePackage` + +This is the semantic deployment artifact description. + +It exists so runtime loading is a projection of semantic packaging decisions, +not a one-off C++ concern. + +Proposed fields: + +```text +GatePackage + gate_id + contract_ref + surface_ref + gate_shape + backend_family + artifact_ref + thresholds + abstain_policy + escalation_policy + trace_schema + version +``` + +### Semantic Relationships + +The first model should follow these relationships: + +```text +DecisionContract + owns high-level decision identity + +DecisionSurface + refines output shape for a contract + +DecisionRequestModel + is evaluated against DecisionContract + DecisionSurface + +DecisionResultModel + must satisfy DecisionSurface constraints + +SuitabilityAssessment + determines whether the decision belongs in RSA at all + +DiagnosisModel + explains weak performance or risky deployment + +ProbeDefinition + declares reusable diagnosis mechanisms + +BackendSuitability + recommends which implementation family should fill an RSA contract + +GatePackage + records the chosen runtime projection of a validated contract +``` + +### Projection Map To The Current C++ Scaffold + +The current scaffold should be treated as the first runtime projection target. + +#### Project directly from semantics into current C++ types + +```text +DecisionContract -> GateDefinition core fields +DecisionSurface -> GateDefinition.labels + candidate_factorizations + future slot fields +DecisionResultModel -> future runtime result structs +SuitabilityAssessment -> GateSuitability +DiagnosisModel -> GateDiagnosis +ProbeDefinition -> Probe metadata + ProbeRequest/ProbeResult contracts +``` + +#### Runtime-only C++ types that should remain projected + +```text +Probe +ProbeRegistry +DiagnosisEngine +SuitabilityAssessor +ConfidenceThresholdProbe +``` + +These are implementations, not semantic source objects. + +### Immediate Type Migration Rule + +Until semantic authoring is fully in place, the current C++ fields should be +interpreted according to this split: + +- semantic-now + - ontology enums + - gate contract fields + - diagnosis enums + - suitability result fields +- runtime-now + - probe interfaces + - engine wiring + - confidence-threshold implementation + - demo executable + +When a new field is proposed, the default question should be: + +- does this describe semantic meaning? + - if yes, add it to the semantic model first +- does this describe runtime execution detail? + - if yes, keep it in projected C++ + +### Immediate Next Conversion Step + +The next concrete implementation step after this document should be: + +1. represent `DecisionContract`, `DecisionSurface`, and `ProbeDefinition` as the + first Whetstone-authored RSA semantic forms +2. map each field in `types.h` into one of those forms +3. mark missing fields needed for structured slot outputs and backend selection +4. treat `types.h` as a temporary projection snapshot until semantic-first + generation replaces hand editing + +## Bootstrap Artifact + +The first machine-readable semantic bootstrap artifact now lives at: + +- `semantic/rsa_semantic_bootstrap_v0.json` + +It is intentionally limited to the first three semantic entities needed to begin +realignment: + +- `DecisionContract` +- `DecisionSurface` +- `ProbeDefinition` + +That file should be treated as the current semantic bootstrap source ahead of any +further handwritten expansion of C++ contract types. diff --git a/semantic/rsa_semantic_bootstrap_v0.json b/semantic/rsa_semantic_bootstrap_v0.json new file mode 100644 index 0000000..2d4a528 --- /dev/null +++ b/semantic/rsa_semantic_bootstrap_v0.json @@ -0,0 +1,490 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "whetstone-rsa/semantic-bootstrap-v0", + "title": "RSA Semantic Bootstrap V0", + "version": "0.1.0-bootstrap", + "status": "bootstrap_source_of_truth", + "notes": [ + "This file is the first machine-readable semantic bootstrap artifact for whetstone_RSA.", + "It is intentionally smaller than the full target semantic model.", + "It defines the first three semantic entities called out in Sprint 002: DecisionContract, DecisionSurface, and ProbeDefinition.", + "Until AST-native authoring is fully in place, this artifact should be treated as the semantic source of truth ahead of handwritten C++ contract growth." + ], + "enums": { + "risk_tier": [ + "low", + "medium", + "high" + ], + "input_structure": [ + "structured", + "unstructured", + "hybrid" + ], + "input_modality": [ + "textual", + "numeric", + "symbolic", + "multimodal" + ], + "output_topology": [ + "discrete", + "continuous", + "hybrid" + ], + "output_constraint": [ + "deterministic", + "bounded", + "open" + ], + "policy_stability": [ + "stable", + "mixed", + "non_stationary" + ], + "gate_shape": [ + "binary", + "multiclass", + "factorized", + "hierarchical" + ], + "backend_family": [ + "linear_model", + "gradient_boosting", + "random_forest", + "bayesian", + "transformer", + "hybrid", + "rl_policy" + ], + "failure_class": [ + "unknown", + "capacity_limited_gate", + "factorizable_gate", + "non_stationary_gate", + "missing_context_gate", + "deterministic_disguised_as_ml_gate", + "label_space_mismatch", + "guardrail_limited_gate" + ], + "intervention": [ + "collect_more_evidence", + "keep_current_gate", + "deploy_with_guardrails", + "raise_confidence_threshold", + "add_retry_with_context", + "add_deterministic_prepass", + "replace_with_deterministic_logic", + "factor_into_subgates", + "switch_to_hierarchical_gate", + "revise_label_schema", + "enrich_input_context", + "evaluate_larger_model_tier" + ] + }, + "entities": { + "DecisionContract": { + "purpose": "Semantic identity of an RSA decision gate, independent of runtime backend choice.", + "fields": [ + { + "name": "gate_id", + "type": "string", + "required": true + }, + { + "name": "task_family", + "type": "string", + "required": true + }, + { + "name": "input_structure", + "type": "enum:input_structure", + "required": true + }, + { + "name": "input_modality", + "type": "enum:input_modality", + "required": true + }, + { + "name": "output_topology", + "type": "enum:output_topology", + "required": true + }, + { + "name": "output_constraint", + "type": "enum:output_constraint", + "required": true + }, + { + "name": "risk_tier", + "type": "enum:risk_tier", + "required": true + }, + { + "name": "policy_stability", + "type": "enum:policy_stability", + "required": true + }, + { + "name": "deterministic_baseline_available", + "type": "boolean", + "required": true + }, + { + "name": "supports_abstain", + "type": "boolean", + "required": true + }, + { + "name": "metadata", + "type": "map", + "required": false + } + ] + }, + "DecisionSurface": { + "purpose": "Semantic description of output shape, factorization, and bounded slot structure for a gate.", + "fields": [ + { + "name": "gate_id", + "type": "string", + "required": true + }, + { + "name": "primary_mode", + "type": "enum", + "allowed_values": [ + "enum", + "structured", + "hybrid" + ], + "required": true + }, + { + "name": "output_labels", + "type": "array", + "required": false + }, + { + "name": "slot_schema", + "type": "array", + "required": false + }, + { + "name": "candidate_gate_shapes", + "type": "array", + "required": true + }, + { + "name": "candidate_factorizations", + "type": "array>", + "required": false + }, + { + "name": "invalid_combinations", + "type": "array", + "required": false + }, + { + "name": "deterministic_prepass_options", + "type": "array", + "required": false + }, + { + "name": "downstream_executor_kind", + "type": "string", + "required": false + } + ] + }, + "ProbeDefinition": { + "purpose": "Semantic declaration of a diagnosis probe before any runtime C++ implementation.", + "fields": [ + { + "name": "probe_id", + "type": "string", + "required": true + }, + { + "name": "applies_to_gate_shapes", + "type": "array", + "required": true + }, + { + "name": "required_contract_fields", + "type": "array", + "required": false + }, + { + "name": "required_evidence_fields", + "type": "array", + "required": false + }, + { + "name": "produced_signals", + "type": "array", + "required": false + }, + { + "name": "failure_hints", + "type": "array", + "required": false + }, + { + "name": "intervention_hints", + "type": "array", + "required": false + } + ] + } + }, + "projection_targets": { + "cpp_runtime": { + "notes": [ + "The current C++ scaffold is a projection target, not the authoring home of the model.", + "Future changes should update this semantic artifact first, then realign projected C++ types." + ], + "field_map": { + "DecisionContract": { + "target_type": "whetstone::rsa::GateDefinition", + "target_fields": [ + "gate_id", + "task_family", + "input_structure", + "input_modality", + "output_topology", + "output_constraint", + "risk_tier", + "label_stability", + "deterministic_baseline_available", + "supports_abstain", + "metadata" + ] + }, + "DecisionSurface": { + "target_type": "whetstone::rsa::GateDefinition", + "target_fields": [ + "labels", + "candidate_factorizations" + ], + "notes": [ + "slot_schema and invalid_combinations are semantic-first fields not yet fully represented in the current C++ scaffold.", + "candidate_gate_shapes and downstream_executor_kind also need future projected support." + ] + }, + "ProbeDefinition": { + "target_type": "whetstone::rsa::Probe plus ProbeRequest/ProbeResult contract", + "target_fields": [ + "probe_id", + "failure_hints", + "intervention_hints" + ], + "notes": [ + "The current runtime does not yet expose semantic-first probe declarations separate from implementations." + ] + } + } + } + }, + "bootstrap_instances": { + "DecisionContract": [ + { + "gate_id": "verification_type", + "task_family": "taskitem_verification_routing", + "input_structure": "unstructured", + "input_modality": "textual", + "output_topology": "discrete", + "output_constraint": "bounded", + "risk_tier": "medium", + "policy_stability": "stable", + "deterministic_baseline_available": true, + "supports_abstain": true, + "metadata": { + "case_study": "whetstone_dsl", + "source_surface": "editor/src/mcp/RegisterArchitectIntakeTools.h", + "inventory_disposition": "rsa_immediate", + "downstream_executor_kind": "verification_contract" + } + }, + { + "gate_id": "worker_type", + "task_family": "taskitem_worker_routing", + "input_structure": "unstructured", + "input_modality": "textual", + "output_topology": "discrete", + "output_constraint": "bounded", + "risk_tier": "medium", + "policy_stability": "mixed", + "deterministic_baseline_available": true, + "supports_abstain": true, + "metadata": { + "case_study": "whetstone_dsl", + "source_surface": "editor/src/TaskitemGeneratorV2.h", + "inventory_disposition": "rsa_immediate", + "downstream_executor_kind": "workflow_routing" + } + }, + { + "gate_id": "prereq_op_selector", + "task_family": "taskitem_execution_readiness", + "input_structure": "hybrid", + "input_modality": "symbolic", + "output_topology": "hybrid", + "output_constraint": "bounded", + "risk_tier": "medium", + "policy_stability": "stable", + "deterministic_baseline_available": true, + "supports_abstain": true, + "metadata": { + "case_study": "whetstone_dsl", + "source_surface": "editor/src/TaskitemGeneratorV2.h", + "inventory_disposition": "rsa_immediate", + "downstream_executor_kind": "execution_contract", + "structured_output_reason": "multi_op_selection" + } + }, + { + "gate_id": "cpp_raising_profile", + "task_family": "cpp_projection_profile_selection", + "input_structure": "structured", + "input_modality": "symbolic", + "output_topology": "discrete", + "output_constraint": "bounded", + "risk_tier": "high", + "policy_stability": "stable", + "deterministic_baseline_available": true, + "supports_abstain": true, + "metadata": { + "case_study": "whetstone_dsl", + "source_surface": "editor/src/mcp/RegisterCppRaisingTools.h", + "inventory_disposition": "rsa_immediate", + "downstream_executor_kind": "cpp_ir_raising" + } + } + ], + "DecisionSurface": [ + { + "gate_id": "verification_type", + "primary_mode": "enum", + "output_labels": [ + "unit", + "schema", + "smoke", + "docs" + ], + "candidate_gate_shapes": [ + "multiclass" + ], + "candidate_factorizations": [], + "deterministic_prepass_options": [ + "target_file_suffix_rules", + "title_keyword_prepass" + ], + "downstream_executor_kind": "verification_policy" + }, + { + "gate_id": "worker_type", + "primary_mode": "enum", + "output_labels": [ + "implementer", + "reviewer", + "architect", + "qa" + ], + "candidate_gate_shapes": [ + "multiclass" + ], + "candidate_factorizations": [], + "deterministic_prepass_options": [ + "title_keyword_rules" + ], + "downstream_executor_kind": "workflow_routing" + }, + { + "gate_id": "prereq_op_selector", + "primary_mode": "structured", + "slot_schema": [ + { + "name": "validate_intake", + "type": "boolean" + }, + { + "name": "architect_review", + "type": "boolean" + }, + { + "name": "resolve_dependencies", + "type": "boolean" + } + ], + "candidate_gate_shapes": [ + "factorized", + "hierarchical" + ], + "candidate_factorizations": [ + [ + "validate_intake", + "architect_review", + "resolve_dependencies" + ] + ], + "invalid_combinations": [], + "deterministic_prepass_options": [ + "always_require_validate_intake" + ], + "downstream_executor_kind": "execution_contract" + }, + { + "gate_id": "cpp_raising_profile", + "primary_mode": "enum", + "output_labels": [ + "safe-first", + "perf-first", + "interop-first" + ], + "candidate_gate_shapes": [ + "multiclass" + ], + "candidate_factorizations": [], + "deterministic_prepass_options": [ + "environment_policy_override", + "ffi_presence_prepass" + ], + "downstream_executor_kind": "cpp_ir_raising" + } + ], + "ProbeDefinition": [ + { + "probe_id": "confidence_threshold_probe", + "applies_to_gate_shapes": [ + "binary", + "multiclass", + "factorized", + "hierarchical" + ], + "required_contract_fields": [ + "supports_abstain" + ], + "required_evidence_fields": [ + "accept_rate", + "accuracy_on_accepted" + ], + "produced_signals": [ + "accept_rate", + "accuracy_on_accepted", + "silent_error_rate" + ], + "failure_hints": [ + "guardrail_limited_gate", + "capacity_limited_gate" + ], + "intervention_hints": [ + "deploy_with_guardrails", + "raise_confidence_threshold", + "evaluate_larger_model_tier" + ] + } + ] + } +} diff --git a/sprints/SPRINT-002-rsa-semantic-bootstrap.md b/sprints/SPRINT-002-rsa-semantic-bootstrap.md new file mode 100644 index 0000000..603683b --- /dev/null +++ b/sprints/SPRINT-002-rsa-semantic-bootstrap.md @@ -0,0 +1,202 @@ +# Sprint 002: RSA Semantic Bootstrap + +## Status + +Planned + +## Intent + +Define the first Whetstone-authored semantic model for `whetstone_RSA` and make it +the new source-of-truth target for future RSA contract evolution. + +This sprint exists to stop the project from continuing as a C++-first library by +inertia. + +## Problem + +`whetstone_RSA` currently has a useful C++ scaffold, but the semantic center of +gravity is still wrong. + +The project has already concluded that RSA should be Whetstone-AST-first and that +the C++ layer should be a runtime projection. That conclusion is documented, but it +is not yet operationalized. + +Right now: + +- the semantic contract is still primarily embodied in `include/whetstone_rsa/types.h` +- diagnosis/runtime interfaces are defined in C++ first +- backend-selection and structured-slot support are planned but not yet represented + in a semantic-first source model + +If that continues, the bootstrap scaffold will become the architecture by default. + +## Scope + +This sprint is about semantic bootstrap only. + +Primary outputs: + +- a first concrete RSA semantic model +- a mapping from the existing C++ scaffold into that semantic model +- a projection boundary definition for future C++ runtime types + +Supporting documents to update if needed: + +- `docs/whetstone_ast_first_integration.md` +- `ROADMAP.md` + +This sprint does **not** require immediate deep changes to `whetstone_DSL` +runtime code. + +## Architectural Constraint + +The semantic source of truth for RSA should live in Whetstone-authored forms. + +The C++ code in `whetstone_RSA` should remain: + +- runtime implementation +- projected contract types +- diagnosis/runtime execution machinery +- backend adapter surface + +It should not remain the primary authoring home of RSA semantics. + +## Sprint Goals + +### 1. Define the first semantic entity set + +Establish the minimum initial semantic model for RSA: + +- `DecisionContract` +- `DecisionSurface` +- `DecisionRequestModel` +- `DecisionResultModel` +- `SuitabilityAssessment` +- `DiagnosisModel` +- `ProbeDefinition` +- `BackendSuitability` +- `GatePackage` + +### 2. Separate semantic contract from runtime behavior + +Classify the current C++ scaffold into: + +- semantic contract entities +- runtime-only execution types +- bootstrap/demo code + +### 3. Define the first projection boundary + +State clearly what will eventually be projected from semantic source into: + +- C++ runtime contract structs +- runtime manifests +- schema artifacts +- benchmark configuration artifacts + +### 4. Prepare version-0 support for both enum and structured-slot gates + +The semantic model must support: + +- bounded enum outputs +- bounded structured outputs with slots + +from the beginning, even if early runtime examples are enum-dominant. + +### 5. Leave RL/bandit as semantic placeholders only + +Include `rl_policy` in backend-family semantics now so the contract stays open, +but keep first implementation effort focused on: + +- classical ML families +- tiny transformer families + +## Concrete Deliverables + +### Deliverable A: Semantic model definition + +Produce a concrete first semantic model with entity names, field groups, and +relationship rules. + +### Deliverable B: C++ scaffold mapping + +Map the current public C++ surface into the semantic model, especially: + +- `types.h` +- `probe.h` +- `probe_registry.h` +- `diagnosis_engine.h` +- `suitability_assessor.h` + +### Deliverable C: Missing-field inventory + +Identify what the current scaffold is missing for: + +- structured slot outputs +- backend suitability +- deployment packaging +- argument-oriented runtime invocation + +### Deliverable D: Projection strategy + +Define how future C++ types should be treated: + +- temporary projection snapshots +- not semantic source +- updated from semantic-first changes rather than evolved independently + +## Acceptance Criteria + +- A first semantic model is explicitly documented in RSA docs +- The model supports both enum and structured-slot gates +- The current C++ scaffold is mapped into semantic vs runtime-only layers +- Backend suitability and gate packaging exist in the semantic model even if not + fully implemented in code +- The roadmap and AST-first integration docs point to the same next step +- A later session could begin implementing semantic-first RSA without re-deriving + the architecture + +## Not In Scope + +- full implementation of semantic authoring inside `whetstone_DSL` +- replacing the current C++ runtime scaffold +- immediate automatic projection generation +- training any model +- wiring RSA into `whetstone_DSL` taskitem generation + +## Case Study Rule + +Use existing RSA C++ types and WhetstoneDSL decision surfaces only as evidence and +bootstrap input. + +Do not let any one case study define RSA semantics by itself. + +## Recommended Execution Order + +1. Inventory the existing C++ public surface. +2. Define the minimum semantic entity set. +3. Map each current C++ field into a semantic or runtime-only bucket. +4. Identify missing fields required by version-0 goals. +5. Define the projection boundary into C++. +6. Record the next implementation step for semantic authoring. + +## Exit Condition + +This sprint is complete when the project no longer needs to answer: + +"What is the first semantic model for RSA, and how does the current C++ scaffold +map into it?" + +That question should be settled well enough that implementation can start from the +semantic model instead of revisiting architecture. + +## Next Step + +Represent the first three RSA semantic entities as Whetstone-authored forms: + +- `DecisionContract` +- `DecisionSurface` +- `ProbeDefinition` + +Then map `include/whetstone_rsa/types.h` onto those entities as a temporary C++ +projection target. diff --git a/sprints/SPRINT-003-whetstone-dsl-gate-semantic-bootstrap.md b/sprints/SPRINT-003-whetstone-dsl-gate-semantic-bootstrap.md new file mode 100644 index 0000000..9745f76 --- /dev/null +++ b/sprints/SPRINT-003-whetstone-dsl-gate-semantic-bootstrap.md @@ -0,0 +1,351 @@ +# Sprint 003: WhetstoneDSL Gate Semantic Bootstrap + +## Status + +Active + +## Intent + +Turn the current RSA architecture and gate inventory into an executable semantic +bootstrap focused on the first concrete `whetstone_DSL` case-study gates. + +This sprint exists to close the gap between: + +- generic RSA architecture +- generic semantic bootstrap language +- actual `whetstone_DSL` decision surfaces + +The output of this sprint should be concrete enough that later work can proceed +without re-deriving what the first real gates are or how they map into semantic +entities. + +## Why This Sprint Exists + +`SPRINT-002-rsa-semantic-bootstrap.md` established the architectural direction, +but it is still too abstract to drive direct implementation. + +We now have a concrete inventory in: + +- `docs/case_studies/whetstone_dsl_gate_inventory.md` + +The next step is to operationalize that inventory into: + +- first concrete case-study gate records +- explicit semantic mapping rules +- clear enum-vs-structured-output boundaries +- acceptance criteria for what counts as a completed bootstrap + +## Scope + +This sprint is still semantic-first. + +Primary outputs: + +- a detailed sprint plan for the first case-study gate set +- semantic records for the first concrete `whetstone_DSL` gates +- mapping notes from `whetstone_DSL` decision surfaces into RSA entities +- explicit skeleton targets for later structured-output gates + +This sprint does **not** require: + +- model training +- wiring RSA runtime into `whetstone_DSL` +- replacing current `whetstone_DSL` heuristics +- projecting full Whetstone-authored AST entities yet + +## Primary Gate Set + +These are the first concrete RSA case-study gates for bootstrap: + +### Immediate RSA gates + +- `verification_type` +- `worker_type` +- `prereq_op_selector` +- `cpp_raising_profile` + +### Skeleton-first semantic decisions + +- `execution_contract_shape` +- `workstream_grouping` +- `target_file_selection` + +## Gate Definitions To Lock Down + +### 1. `verification_type` + +Source surface: + +- `RegisterArchitectIntakeTools.h` + +Decision form: + +- bounded enum + +Expected outputs: + +- `unit` +- `schema` +- `docs` +- `smoke` + +Contract interpretation: + +- unstructured textual input +- bounded discrete output +- stable policy +- deterministic downstream verification behavior + +### 2. `worker_type` + +Source surface: + +- `TaskitemGeneratorV2.h` + +Decision form: + +- bounded enum + +Expected outputs: + +- `implementer` +- `reviewer` +- `architect` +- `qa` + +Contract interpretation: + +- unstructured or hybrid textual input +- bounded discrete output +- mixed context sensitivity +- deterministic downstream routing behavior + +### 3. `prereq_op_selector` + +Source surface: + +- `TaskitemGeneratorV2.h` + +Decision form: + +- bounded structured output +- factorized / multi-label candidate + +Expected atomic outputs: + +- `validate-intake` +- `architect-review` +- `resolve-dependencies` + +Contract interpretation: + +- hybrid semantic gate +- not a good flat multiclass default +- primary probe target for factorization analysis + +### 4. `cpp_raising_profile` + +Source surface: + +- `RegisterCppRaisingTools.h` + +Decision form: + +- bounded enum + +Expected outputs: + +- `safe-first` +- `perf-first` +- `interop-first` + +Contract interpretation: + +- structured symbolic input +- bounded discrete output +- deterministic downstream execution and artifact generation + +## Skeleton Decisions To Represent But Not Train Yet + +### `execution_contract_shape` + +Why skeleton first: + +- output is structured +- current semantics are implicit across many heuristics +- not yet stable enough as a direct training target + +Required semantic fields: + +- target files +- required tools +- acceptance commands +- step sequence +- cross-project targets +- resource locks + +### `workstream_grouping` + +Why skeleton first: + +- current logic mixes lexical grouping with semantic bundling +- good future structured-output gate +- not yet clean enough for direct benchmark comparisons + +### `target_file_selection` + +Why skeleton first: + +- current logic is bounded but still heavily heuristic +- needs clearer contract and evaluation rules before training + +## Deliverables + +### Deliverable A: Case-study gate records + +Update the semantic bootstrap so it explicitly records: + +- gate identity +- ontology fields +- output labels or slot schema +- candidate gate shape +- deterministic baseline availability +- downstream executor kind + +for the four immediate gates. + +Status: + +- complete in `semantic/rsa_semantic_bootstrap_v0.json` + +### Deliverable B: Mapping rules + +Write down the mapping from current `whetstone_DSL` code surfaces into RSA +semantic entities: + +- `TaskitemGeneratorV2` +- `TaskitemConfidenceAmbiguity` +- `RegisterArchitectIntakeTools` +- `RegisterCppRaisingTools` +- `RoutingEngine` + +Status: + +- complete in `docs/case_studies/whetstone_dsl_semantic_mapping.md` + +### Deliverable C: Structured-output bootstrap boundary + +Define how `prereq_op_selector` and `execution_contract_shape` are represented +without collapsing them into flat labels. + +Status: + +- partially complete +- `execution_contract_shape` skeleton captured in + `docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md` +- extraction rules for `prereq_op_selector` and execution-contract child + decisions captured in + `docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md` + +### Deliverable D: Artifact-grounded extraction path + +Define which surviving `whetstone_DSL` corpora can support RSA-aligned +extraction, and which should remain context or evaluation corpora. + +Status: + +- partially complete +- audit captured in `docs/case_studies/whetstone_dsl_run_artifact_audit.md` +- first extraction rules captured in + `docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md` + +## Active Next Task + +Turn the first extraction specs into machine-readable extraction contracts for: + +- `prereq_op_selector` +- `target_file_selection` +- `acceptance_command_selection` +- `required_tool_selection` + +Status: + +- partially complete +- `prereq_op_selector` is now represented as structured/factorized in + `semantic/rsa_semantic_bootstrap_v0.json` +- `execution_contract_shape` skeleton is now defined in + `docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md` +- `target_file_selection` and `workstream_grouping` still need explicit skeleton field sets + +### Deliverable D: Probe-readiness notes + +Record which probes should apply first to each gate: + +- `confidence_threshold_probe` +- `factorization_probe` +- `schema_stability_probe` +- `deterministic_rule_probe` +- `context_width_probe` + +Status: + +- partially complete +- immediate-gate probe candidates are recorded in the gate inventory and + bootstrap notes +- still needs a single consolidated probe-readiness artifact + +## Recommended Execution Order + +1. Extend the semantic bootstrap artifact with the first four case-study gates. +2. Record mapping notes from each current `whetstone_DSL` implementation surface. +3. Represent `prereq_op_selector` as structured/factorized rather than flat. +4. Represent skeleton-only decisions for execution-contract shaping and grouping. +5. Record which current gates should remain deterministic. +6. Leave runtime wiring and training for later sprints. + +Current progress: + +- steps 1, 2, and 3 are complete +- `execution_contract_shape` is complete enough for first-pass skeleton treatment +- `target_file_selection` is now the active task + +## Acceptance Criteria + +- the sprint names a fixed first gate set rather than speaking about gates in the abstract +- the semantic bootstrap artifact contains explicit records for: + - `verification_type` + - `worker_type` + - `prereq_op_selector` + - `cpp_raising_profile` +- each gate is classified as enum, structured, or factorized +- at least three skeleton-first decision entities are identified and justified +- deterministic-baseline gates are explicitly separated from RSA gates +- the sprint is actionable without reopening architecture debate + +## Not In Scope + +- training a new specialist +- modifying `whetstone_DSL` runtime code +- adding new RSA C++ runtime classes unless needed for semantic alignment +- making Whetstone AST the full live authoring source this sprint + +## Exit Condition + +This sprint is complete when a later session can answer all of the following +without ambiguity: + +- what are the first real `whetstone_DSL` gates in RSA terms +- which are enum versus structured-output +- which should remain deterministic +- which structured decisions need skeleton treatment before training +- what the first semantic bootstrap records look like + +## Immediate First Task + +Extend `semantic/rsa_semantic_bootstrap_v0.json` with a case-study gate section +covering: + +- `verification_type` +- `worker_type` +- `prereq_op_selector` +- `cpp_raising_profile`