whetstone_RSA/sprints/SPRINT-003-whetstone-dsl-gate-semantic-bootstrap.md

Sprint 003: WhetstoneDSL Gate Semantic Bootstrap

Status

Active — pending intake contract design (see Architectural Constraint below)

Intent

Turn the current RSA architecture and gate inventory into an executable semantic bootstrap focused on the first concrete whetstone_DSL case-study gates.

This sprint exists to close the gap between:

• generic RSA architecture
• generic semantic bootstrap language
• actual whetstone_DSL decision surfaces

The output of this sprint should be concrete enough that later work can proceed without re-deriving what the first real gates are or how they map into semantic entities.

Architectural Constraint — Intake Contract

Discovered this session. This constraint changes how RSA training data is sourced.

The current whetstone_DSL run corpus predates a planned intake contract. The intended design is that external tools (HiveMind, gleaner, constcad, etc.) submit project descriptions in a structured contract format that explicitly carries the fields needed to make RSA gate decisions bounded and classifiable.

Without the contract, the intake produces ~18 unique template outputs regardless of project. With the contract, every project submission becomes a varied, labeled training example with known decision-relevant fields.

Correct pipeline (with contract):

External tool
    ↓  submits structured project description (contract fields)
Intake contract schema  ← explicit per-field semantics
    ↓
architect_intake  reads contract, produces normalized form
    ↓
RSA gates fire on contract fields  ← bounded, varied, discriminative
    ↓
generate_taskitems  ← deterministic given gate outputs

Implications for RSA design:

• Gate input features are contract fields, not free-form text
• Training data comes from contract submissions, not ad hoc specs
• The current corpus is valid for label schema validation only
• Sprint plan step descriptions remain a valid interim text source for baseline training
• The contract schema must be defined before RSA training data can be collected at scale

Next design milestone (not this sprint):

Define the intake contract schema: what fields must an external tool provide, and which fields are the discriminative inputs for each immediate RSA gate (verification_type, worker_type, prereq_op_selector, cpp_raising_profile).

Why This Sprint Exists

SPRINT-002-rsa-semantic-bootstrap.md established the architectural direction, but it is still too abstract to drive direct implementation.

We now have a concrete inventory in:

• docs/case_studies/whetstone_dsl_gate_inventory.md

The next step is to operationalize that inventory into:

• first concrete case-study gate records
• explicit semantic mapping rules
• clear enum-vs-structured-output boundaries
• acceptance criteria for what counts as a completed bootstrap

Scope

This sprint is still semantic-first.

Primary outputs:

• a detailed sprint plan for the first case-study gate set
• semantic records for the first concrete whetstone_DSL gates
• mapping notes from whetstone_DSL decision surfaces into RSA entities
• explicit skeleton targets for later structured-output gates

This sprint does not require:

• model training
• wiring RSA runtime into whetstone_DSL
• replacing current whetstone_DSL heuristics
• projecting full Whetstone-authored AST entities yet

Primary Gate Set

These are the first concrete RSA case-study gates for bootstrap:

Immediate RSA gates

• verification_type
• worker_type
• prereq_op_selector
• cpp_raising_profile

Skeleton-first semantic decisions

• execution_contract_shape
• workstream_grouping
• target_file_selection

Gate Definitions To Lock Down

1. verification_type

Source surface:

• RegisterArchitectIntakeTools.h

Decision form:

• bounded enum

Expected outputs:

• unit
• schema
• docs
• smoke

Contract interpretation:

• unstructured textual input
• bounded discrete output
• stable policy
• deterministic downstream verification behavior

2. worker_type

Source surface:

• TaskitemGeneratorV2.h

Decision form:

• bounded enum

Expected outputs:

• implementer
• reviewer
• architect
• qa

Contract interpretation:

• unstructured or hybrid textual input
• bounded discrete output
• mixed context sensitivity
• deterministic downstream routing behavior

3. prereq_op_selector

Source surface:

• TaskitemGeneratorV2.h

Decision form:

• bounded structured output
• factorized / multi-label candidate

Expected atomic outputs:

• validate-intake
• architect-review
• resolve-dependencies

Contract interpretation:

• hybrid semantic gate
• not a good flat multiclass default
• primary probe target for factorization analysis

4. cpp_raising_profile

Source surface:

• RegisterCppRaisingTools.h

Decision form:

• bounded enum

Expected outputs:

• safe-first
• perf-first
• interop-first

Contract interpretation:

• structured symbolic input
• bounded discrete output
• deterministic downstream execution and artifact generation

Skeleton Decisions To Represent But Not Train Yet

execution_contract_shape

Why skeleton first:

• output is structured
• current semantics are implicit across many heuristics
• not yet stable enough as a direct training target

Required semantic fields:

• target files
• required tools
• acceptance commands
• step sequence
• cross-project targets
• resource locks

workstream_grouping

Why skeleton first:

• current logic mixes lexical grouping with semantic bundling
• good future structured-output gate
• not yet clean enough for direct benchmark comparisons

target_file_selection

Why skeleton first:

• current logic is bounded but still heavily heuristic
• needs clearer contract and evaluation rules before training

Deliverables

Deliverable A: Case-study gate records

Update the semantic bootstrap so it explicitly records:

• gate identity
• ontology fields
• output labels or slot schema
• candidate gate shape
• deterministic baseline availability
• downstream executor kind

for the four immediate gates.

Status:

• complete in semantic/rsa_semantic_bootstrap_v0.json

Deliverable B: Mapping rules

Write down the mapping from current whetstone_DSL code surfaces into RSA semantic entities:

• TaskitemGeneratorV2
• TaskitemConfidenceAmbiguity
• RegisterArchitectIntakeTools
• RegisterCppRaisingTools
• RoutingEngine

Status:

• complete in docs/case_studies/whetstone_dsl_semantic_mapping.md

Deliverable C: Structured-output bootstrap boundary

Define how prereq_op_selector and execution_contract_shape are represented without collapsing them into flat labels.

Status:

• partially complete
• execution_contract_shape skeleton captured in docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md
• extraction rules for prereq_op_selector and execution-contract child decisions captured in docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md
• machine-readable extraction contracts captured in semantic/rsa_gate_extraction_contracts_v0.json

Deliverable D: Artifact-grounded extraction path

Define which surviving whetstone_DSL corpora can support RSA-aligned extraction, and which should remain context or evaluation corpora.

Status:

• partially complete
• audit captured in docs/case_studies/whetstone_dsl_run_artifact_audit.md
• first extraction rules captured in docs/case_studies/whetstone_dsl_gate_extraction_specs_v0.md
• parser-facing extraction contracts captured in semantic/rsa_gate_extraction_contracts_v0.json

Active Next Task

Use semantic/rsa_gate_extraction_contracts_v0.json to draft the first artifact parser for:

• prereq_op_selector
• target_file_selection
• acceptance_command_selection
• required_tool_selection

Status:

• complete
• parser implemented in tools/extract_gate_rows.py
• sample output generated in semantic/extracted_gate_rows_sample.ndjson

Next Active Task

Run the parser over a broader taskitem-run slice and review extraction yield by:

• gate id
• provenance class
• rejection-flag frequency

Status:

• complete for first 50-run slice
• output captured in semantic/extracted_gate_rows_slice.ndjson
• yield review captured in docs/case_studies/whetstone_dsl_extraction_yield_snapshot.md

Updated Working Conclusion

The first extraction-ready dataset candidate is:

• prereq_op_selector

The following remain explicit but review-gated due to strong sibling-uniform template behavior in the current slice:

• target_file_selection
• acceptance_command_selection
• required_tool_selection

Next Active Task

Turn prereq_op_selector into the first accepted dataset slice, with:

• accepted rows
• rejected rows
• explicit schema-drift examples

Status:

• complete for the 50-run slice
• split produced by tools/split_prereq_op_dataset.py
• outputs at semantic/prereq_op_dataset/
  • accepted.ndjson — 94 rows
  • rejected.ndjson — 12 rows (all schema-drift)
  • schema_drift.ndjson — 12 rows (same set; all rejections are schema-drift)

Key Findings From Split

Label distribution (accepted rows):

Combo	Count
validate_intake only	13
validate_intake + resolve_dependencies	47
validate_intake + architect_review	34

needs_validate_intake is always True across all 94 accepted rows.

This means:

• validate_intake is not a decision surface — it is a constant output in this corpus.
• The effective gate is a 3-way classification:
  • validate_only (13 rows, 13.8%)
  • validate+resolve (47 rows, 50%)
  • validate+architect (34 rows, 36.2%)
• Or equivalently, two independent binary decisions:
  • needs_resolve_dependencies (47/94 = 50%)
  • needs_architect_review (34/94 = 36%)

Schema-drift rows carry valid label signal. The 12 rejected rows have correct needs_* booleans. The only issue is source_values contains tool IDs mixed with valid op names. Salvage path: re-extract only the valid op names and strip the tool IDs, then accept the row.

Decision Required Before Broadening

Before running over more runs, decide:

1. Hard-filter or salvage schema-drift rows?

   • Hard-filter: discard 12 rows, keep extraction simple.
   • Salvage: strip invalid values from source_values, promote to accepted.
   • Recommendation: hard-filter for the first training run. Salvage later if the accepted corpus stays small after broadening.

2. Flat 3-way vs factorized binary formulation?

   • Flat 3-way: treat as single multiclass with 3 outputs (ignoring validate_intake).
   • Factorized binary: two independent classifiers for architect_review and resolve_dependencies. validate_intake is dropped as a constant.
   • Recommendation: factorized binary. The two decisions appear semantically orthogonal in the data.

1000-Run Extraction Results

Output at semantic/prereq_op_dataset_1000/ (1000 runs, 1321 tasks):

• accepted: 1297 rows
• rejected: 24 rows (all schema-drift)
• schema drift variants:
  • tool-ID drift (whetstone_generate_taskitems, etc.): 12 rows
  • manual-approval drift: 12 rows (undocumented op name in prerequisiteOps)

Label distribution (1000-run accepted):

Combo	Count	Pct
validate_intake + resolve_dependencies	996	76.8%
validate_intake + architect_review	256	19.7%
validate_intake only	44	3.4%
all three	1	0.1%

needs_validate_intake remains always True at 1000-run scale.

Effective gate at 1000 runs:

• needs_resolve_dependencies: 997/1297 = 76.9% positive
• needs_architect_review: 257/1297 = 19.8% positive

Class imbalance for needs_architect_review (~80/20) is manageable for training. The validate_intake constant can be dropped from the label space.

Both schema-drift variants are salvageable: valid bits (needs_*) are correct; only source_values contains noise. Recommendation: hard-filter for first training run, evaluate whether 24 extra rows are worth a salvage pass after.

Text Feature Audit — Critical Finding

Attempted text composition from extracted row context fields: title, reasons, normalized_requirements. Result:

• unique texts across 1297 rows: 17 (title+reasons), 18 (acceptance criteria)
• top text appears 697 times ("Execution Readiness Primary | high_confidence_clear_path")
• structured features (confidence, dependency count) are also nearly constant

Root cause: The whetstone_DSL corpus uses a small set of canonical intake specs (~18 unique templates) run many times. All tasks generated from the same intake share the same context text. The step description text — the actual discriminative feature — is not stored in run artifacts.

Conclusion:

The run corpus captures labels correctly but not the text features needed to train a text-based prereq_op specialist.

The correct training data source for text input remains sprint plan step descriptions (as used by gen_prereq_op_data.py in the specialist fleet). The run corpus contributes:

• validated label schema
• label frequency distribution (validate-intake constant, resolve 77%, architect 20%)
• schema-drift pattern documentation
• corpus for future label-correction of heuristic sprint-plan data

Data strategy going forward:

Source	Role
Sprint plan step descriptions	Primary text input for training
Run corpus (accepted.ndjson)	Label validation and frequency baseline
Run corpus schema-drift rows	Schema-drift taxonomy for robustness testing

The sprint plan approach is already implemented in: CLionProjects/whetstone_DSL/specialists/scripts/gen_prereq_op_data.py

That script uses keyword heuristics to assign labels. The RSA extraction work validates that the heuristic label schema is correct; improving label accuracy requires linking sprint plan steps back to run artifacts (not yet feasible).

Next Active Task

Proceed to training using existing sprint plan data with the factorized binary formulation:

1. Run gen_prereq_op_data.py to regenerate sprint plan TSVs.
2. Split into two binary TSVs (resolve / architect) using prereq_op_binary_split.py.
3. Train both specialists with train_specialist_pt.py.
4. Evaluate combined accuracy against the 4-way eval set.

The RSA-extracted accepted.ndjson dataset remains available as a label-validation reference and can be used for post-training audit (do sprint-plan-trained specialists generalize to real run labels?).

• execution_contract_shape skeleton is now defined in docs/case_studies/whetstone_dsl_execution_contract_shape_skeleton.md
• target_file_selection and workstream_grouping still need explicit skeleton field sets

Deliverable D: Probe-readiness notes

Record which probes should apply first to each gate:

• confidence_threshold_probe
• factorization_probe
• schema_stability_probe
• deterministic_rule_probe
• context_width_probe

Status:

• partially complete
• immediate-gate probe candidates are recorded in the gate inventory and bootstrap notes
• still needs a single consolidated probe-readiness artifact

Recommended Execution Order

1. Extend the semantic bootstrap artifact with the first four case-study gates.
2. Record mapping notes from each current whetstone_DSL implementation surface.
3. Represent prereq_op_selector as structured/factorized rather than flat.
4. Represent skeleton-only decisions for execution-contract shaping and grouping.
5. Record which current gates should remain deterministic.
6. Leave runtime wiring and training for later sprints.

Current progress:

• steps 1, 2, and 3 are complete
• execution_contract_shape is complete enough for first-pass skeleton treatment
• target_file_selection is now the active task

Acceptance Criteria

• the sprint names a fixed first gate set rather than speaking about gates in the abstract
• the semantic bootstrap artifact contains explicit records for:
  • verification_type
  • worker_type
  • prereq_op_selector
  • cpp_raising_profile
• each gate is classified as enum, structured, or factorized
• at least three skeleton-first decision entities are identified and justified
• deterministic-baseline gates are explicitly separated from RSA gates
• the sprint is actionable without reopening architecture debate

Not In Scope

• training a new specialist
• modifying whetstone_DSL runtime code
• adding new RSA C++ runtime classes unless needed for semantic alignment
• making Whetstone AST the full live authoring source this sprint

Exit Condition

This sprint is complete when a later session can answer all of the following without ambiguity:

• what are the first real whetstone_DSL gates in RSA terms
• which are enum versus structured-output
• which should remain deterministic
• which structured decisions need skeleton treatment before training
• what the first semantic bootstrap records look like

Immediate First Task

Extend semantic/rsa_semantic_bootstrap_v0.json with a case-study gate section covering:

• verification_type
• worker_type
• prereq_op_selector
• cpp_raising_profile