whetstone_RSA/sprints/SPRINT-003-whetstone-dsl-gate-semantic-bootstrap.md

# Sprint 003: WhetstoneDSL Gate Semantic Bootstrap

## Status

Active

## Intent

Turn the current RSA architecture and gate inventory into an executable semantic
bootstrap focused on the first concrete `whetstone_DSL` case-study gates.

This sprint exists to close the gap between:

- generic RSA architecture
- generic semantic bootstrap language
- actual `whetstone_DSL` decision surfaces

The output of this sprint should be concrete enough that later work can proceed
without re-deriving what the first real gates are or how they map into semantic
entities.

## Why This Sprint Exists

`SPRINT-002-rsa-semantic-bootstrap.md` established the architectural direction,
but it is still too abstract to drive direct implementation.

We now have a concrete inventory in:

- `docs/case_studies/whetstone_dsl_gate_inventory.md`

The next step is to operationalize that inventory into:

- first concrete case-study gate records
- explicit semantic mapping rules
- clear enum-vs-structured-output boundaries
- acceptance criteria for what counts as a completed bootstrap

## Scope

This sprint is still semantic-first.

Primary outputs:

- a detailed sprint plan for the first case-study gate set
- semantic records for the first concrete `whetstone_DSL` gates
- mapping notes from `whetstone_DSL` decision surfaces into RSA entities
- explicit skeleton targets for later structured-output gates

This sprint does **not** require:

- model training
- wiring RSA runtime into `whetstone_DSL`
- replacing current `whetstone_DSL` heuristics
- projecting full Whetstone-authored AST entities yet

## Primary Gate Set

These are the first concrete RSA case-study gates for bootstrap:

### Immediate RSA gates

- `verification_type`
- `worker_type`
- `prereq_op_selector`
- `cpp_raising_profile`

### Skeleton-first semantic decisions

- `execution_contract_shape`
- `workstream_grouping`
- `target_file_selection`

## Gate Definitions To Lock Down

### 1. `verification_type`

Source surface:

- `RegisterArchitectIntakeTools.h`

Decision form:

- bounded enum

Expected outputs:

- `unit`
- `schema`
- `docs`
- `smoke`

Contract interpretation:

- unstructured textual input
- bounded discrete output
- stable policy
- deterministic downstream verification behavior

### 2. `worker_type`

Source surface:

- `TaskitemGeneratorV2.h`

Decision form:

- bounded enum

Expected outputs:

- `implementer`
- `reviewer`
- `architect`
- `qa`

Contract interpretation:

- unstructured or hybrid textual input
- bounded discrete output
- mixed context sensitivity
- deterministic downstream routing behavior

### 3. `prereq_op_selector`

Source surface:

- `TaskitemGeneratorV2.h`

Decision form:

- bounded structured output
- factorized / multi-label candidate

Expected atomic outputs:

- `validate-intake`
- `architect-review`
- `resolve-dependencies`

Contract interpretation:

- hybrid semantic gate
- not a good flat multiclass default
- primary probe target for factorization analysis

### 4. `cpp_raising_profile`

Source surface:

- `RegisterCppRaisingTools.h`

Decision form:

- bounded enum

Expected outputs:

- `safe-first`
- `perf-first`
- `interop-first`

Contract interpretation:

- structured symbolic input
- bounded discrete output
- deterministic downstream execution and artifact generation

## Skeleton Decisions To Represent But Not Train Yet

### `execution_contract_shape`

Why skeleton first:

- output is structured
- current semantics are implicit across many heuristics
- not yet stable enough as a direct training target

Required semantic fields:

- target files
- required tools
- acceptance commands
- step sequence
- cross-project targets
- resource locks

### `workstream_grouping`

Why skeleton first:

- current logic mixes lexical grouping with semantic bundling
- good future structured-output gate
- not yet clean enough for direct benchmark comparisons

### `target_file_selection`

Why skeleton first:

- current logic is bounded but still heavily heuristic
- needs clearer contract and evaluation rules before training

## Deliverables

### Deliverable A: Case-study gate records

Update the semantic bootstrap so it explicitly records:

- gate identity
- ontology fields
- output labels or slot schema
- candidate gate shape
- deterministic baseline availability
- downstream executor kind

for the four immediate gates.

Status:

- complete in `semantic/rsa_semantic_bootstrap_v0.json`

### Deliverable B: Mapping rules

Write down the mapping from current `whetstone_DSL` code surfaces into RSA
semantic entities:

- `TaskitemGeneratorV2`
- `TaskitemConfidenceAmbiguity`
- `RegisterArchitectIntakeTools`
- `RegisterCppRaisingTools`
- `RoutingEngine`

Status:

- complete in `docs/case_studies/whetstone_dsl_semantic_mapping.md`

### Deliverable C: Structured-output bootstrap boundary

Define how `prereq_op_selector` and `execution_contract_shape` are represented
without collapsing them into flat labels.

Status:

- partially complete
- `execution_contract_shape` skeleton captured in
  `docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md`
- extraction rules for `prereq_op_selector` and execution-contract child
  decisions captured in
  `docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md`
- machine-readable extraction contracts captured in
  `semantic/rsa_gate_extraction_contracts_v0.json`

### Deliverable D: Artifact-grounded extraction path

Define which surviving `whetstone_DSL` corpora can support RSA-aligned
extraction, and which should remain context or evaluation corpora.

Status:

- partially complete
- audit captured in `docs/case_studies/whetstone_dsl_run_artifact_audit.md`
- first extraction rules captured in
  `docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md`
- parser-facing extraction contracts captured in
  `semantic/rsa_gate_extraction_contracts_v0.json`

## Active Next Task

Use `semantic/rsa_gate_extraction_contracts_v0.json` to draft the first
artifact parser for:

- `prereq_op_selector`
- `target_file_selection`
- `acceptance_command_selection`
- `required_tool_selection`

Status:

- complete
- parser implemented in `tools/extract_gate_rows.py`
- sample output generated in `semantic/extracted_gate_rows_sample.ndjson`

## Next Active Task

Run the parser over a broader taskitem-run slice and review extraction yield by:

- gate id
- provenance class
- rejection-flag frequency

Status:

- complete for first 50-run slice
- output captured in `semantic/extracted_gate_rows_slice.ndjson`
- yield review captured in
  `docs/case_studies/whetstone_dsl_extraction_yield_snapshot.md`

## Updated Working Conclusion

The first extraction-ready dataset candidate is:

- `prereq_op_selector`

The following remain explicit but review-gated due to strong sibling-uniform
template behavior in the current slice:

- `target_file_selection`
- `acceptance_command_selection`
- `required_tool_selection`

## Next Active Task

Turn `prereq_op_selector` into the first accepted dataset slice, with:

- accepted rows
- rejected rows
- explicit schema-drift examples

Status:

- partially complete
- `prereq_op_selector` is now represented as structured/factorized in
  `semantic/rsa_semantic_bootstrap_v0.json`
- `execution_contract_shape` skeleton is now defined in
  `docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md`
- `target_file_selection` and `workstream_grouping` still need explicit skeleton field sets

### Deliverable D: Probe-readiness notes

Record which probes should apply first to each gate:

- `confidence_threshold_probe`
- `factorization_probe`
- `schema_stability_probe`
- `deterministic_rule_probe`
- `context_width_probe`

Status:

- partially complete
- immediate-gate probe candidates are recorded in the gate inventory and
  bootstrap notes
- still needs a single consolidated probe-readiness artifact

## Recommended Execution Order

1. Extend the semantic bootstrap artifact with the first four case-study gates.
2. Record mapping notes from each current `whetstone_DSL` implementation surface.
3. Represent `prereq_op_selector` as structured/factorized rather than flat.
4. Represent skeleton-only decisions for execution-contract shaping and grouping.
5. Record which current gates should remain deterministic.
6. Leave runtime wiring and training for later sprints.

Current progress:

- steps 1, 2, and 3 are complete
- `execution_contract_shape` is complete enough for first-pass skeleton treatment
- `target_file_selection` is now the active task

## Acceptance Criteria

- the sprint names a fixed first gate set rather than speaking about gates in the abstract
- the semantic bootstrap artifact contains explicit records for:
  - `verification_type`
  - `worker_type`
  - `prereq_op_selector`
  - `cpp_raising_profile`
- each gate is classified as enum, structured, or factorized
- at least three skeleton-first decision entities are identified and justified
- deterministic-baseline gates are explicitly separated from RSA gates
- the sprint is actionable without reopening architecture debate

## Not In Scope

- training a new specialist
- modifying `whetstone_DSL` runtime code
- adding new RSA C++ runtime classes unless needed for semantic alignment
- making Whetstone AST the full live authoring source this sprint

## Exit Condition

This sprint is complete when a later session can answer all of the following
without ambiguity:

- what are the first real `whetstone_DSL` gates in RSA terms
- which are enum versus structured-output
- which should remain deterministic
- which structured decisions need skeleton treatment before training
- what the first semantic bootstrap records look like

## Immediate First Task

Extend `semantic/rsa_semantic_bootstrap_v0.json` with a case-study gate section
covering:

- `verification_type`
- `worker_type`
- `prereq_op_selector`
- `cpp_raising_profile`