whetstone_RSA/docs/case_studies/whetstone_dsl_extraction_yield_snapshot.md

# WhetstoneDSL Extraction Yield Snapshot

## Slice

Parser:

- `tools/extract_gate_rows.py`

Contracts:

- `semantic/rsa_gate_extraction_contracts_v0.json`

Input slice:

- first 50 run directories under
  `CLionProjects/whetstone_DSL/logs/taskitem_runs`

Output:

- `semantic/extracted_gate_rows_slice.ndjson`

## Summary

Observed counts from the 50-run slice:

- runs processed: `50`
- tasks processed: `106`
- rows emitted: `424`
- rows with rejection flags: `330`

Per-gate emitted rows:

- `prereq_op_selector`: `106`
- `target_file_selection`: `106`
- `acceptance_command_selection`: `106`
- `required_tool_selection`: `106`

Per-gate rejected rows:

- `prereq_op_selector`: `12`
- `target_file_selection`: `106`
- `acceptance_command_selection`: `106`
- `required_tool_selection`: `106`

## Dominant Rejection Flags

- `sibling_uniform_target_files`: `106`
- `sibling_uniform_acceptance_commands`: `106`
- `sibling_uniform_required_tools`: `106`
- `invalid_prerequisite_values:whetstone_generate_taskitems,whetstone_queue_ready,whetstone_validate_taskitem`: `12`

## Interpretation

### `prereq_op_selector`

This is the strongest current extraction surface.

It survives most of the slice cleanly, and the rejection cases are informative
rather than noisy. The rejected rows come from schema-drift runs where
`prerequisiteOps` was contaminated with tool ids such as:

- `whetstone_generate_taskitems`
- `whetstone_queue_ready`
- `whetstone_validate_taskitem`

That means the parser is catching real label pollution rather than inventing
false negatives.

### `target_file_selection`

Present explicitly, but not yet trustworthy as broad supervision in this slice.

The dominant issue is sibling-uniform output: all tasks in a run often carry the
same target-file list, which makes naive extraction look more like template
reuse than local bounded choice.

### `acceptance_command_selection`

Present explicitly, but also highly template-like in this slice.

The same sibling-uniform pattern appears across runs, which suggests these rows
should remain review-gated until we either:

- find more diverse artifact families
- or factor the decision more tightly

### `required_tool_selection`

Same current status as acceptance commands.

The field exists and is easy to parse, but the observed slice suggests strong
template carryover rather than high-confidence local decision variance.

## Immediate Conclusion

The current first supervised RSA gate should be:

- `prereq_op_selector`

The following remain promising, but should stay review-gated or filtered more
aggressively before broader extraction:

- `target_file_selection`
- `acceptance_command_selection`
- `required_tool_selection`

## Next Steps

1. Tighten parser-side filtering for schema-drift prereq rows and optionally
   split accepted vs rejected outputs.

2. Search for artifact families with higher local variance for:
   - `target_file_selection`
   - `acceptance_command_selection`
   - `required_tool_selection`

3. Promote `prereq_op_selector` to the first extraction-ready gate dataset.