bill/whetstone_DSL
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add raw structural projector before candidate scoring (sprint 263)

Browse Source
This commit is contained in:
Bill
2026-02-26 16:59:28 -07:00
parent f534d7f0fa
commit fe85f1b515
8 changed files with 249 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/constructive_editing_runtime_plan.md
View File

@@ -247,3 +247,7 @@ Planning runtime controls (active):
- `WSTONE_NATIVE_RAW_TEMPLATE_CONTROL_PACK`
- `WSTONE_NATIVE_RAW_TEMPLATE_CONTROL_MAX_SIGNALS`
- `native_raw_template_control_pack`
- raw candidate scoring can evaluate deterministic structural projections pre-score:
- `tools/mcp/project_raw_candidate_structure.py`
- `WSTONE_NATIVE_RAW_STRUCTURAL_PROJECTOR`
- `native_raw_structural_projector`

2
docs/generator_readiness_gap_registry_2026-02-26.md
View File

@@ -47,7 +47,7 @@ This is the canonical dated registry for "not production-ready" generator gaps.
| GR-018 | Performance/security/rollout constrained refactor enforcement gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/results.jsonl` | `partial` | Hard checks now enforce migration rollback + data-loss policy, security deny-by-default, SLO p95 presence, and rollout staged+abort policy. Enforcement is contract-level; generator capability under these constraints is still weak in C++ AB path. | Constraint-aware generation follow-up |
| GR-019 | Parity-blocked readiness load (gating without capability closure) | `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r7/summary.json`, `logs/taskitem_runs/challenging_subset_prod_20260226_r7/summary.json`, `docs/sprint225_227_execution_tracker_2026-02-26.md` | `partial` | Sprint 225-227 reduced blocked parity load from `6` to `0` on both tracked hard catalogs while keeping unresolved divergence at `0`. This closes immediate safety debt for current corpora, but robustness is still contingent on pattern-driven repair classes. | Generalize repairs beyond queue-shaped transpile outputs |
| GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`, `logs/taskitem_runs/TEST_ONLY_sprint239_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145631/00_summary.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md`, `docs/sprint238_execution_tracker_2026-02-26.md`, `docs/sprint239_execution_tracker_2026-02-26.md`, `docs/sprint240_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Sprint 238 added native decomposition hard gate in pipeline (`min task count`, `min semantic signal count`) so weak native generation can be blocked before fallback masking. Sprint 239 added native reason enrichment and improved semantic signal density (`0 -> 6`). Sprint 240 added native decomposition retry with explicit minimum-task policy; on current sample retry attempted but did not improve depth (`2 -> 2`). | Native decomposition task-depth upgrade (increase native task granularity beyond 2) |
| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_20260226_151651/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_enforce_20260226_151709/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint245_intrinsic_20260226_151835/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_20260226_153033/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_enforce_20260226_153045/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_20260226_153230/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_enforce_20260226_153241/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint248_rawsearch_20260226_153903/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint249_rawguard_20260226_154028/02ae_raw_candidate_search.json`, `logs/taskitem_runs/TEST_ONLY_sprint250_closure_ladder_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint253_closure_ladder_batch_20260226/closure_ladder_batch_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint254_closure_ladder_batch_20260226_r2/raw_gap_backlog.json`, `logs/taskitem_runs/TEST_ONLY_sprint255_rawharden_20260226_161924/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162423/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162633/02ae_raw_candidate_search.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/01e_raw_top_gap_requirements_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163148/02af_raw_adaptive_retry_score.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163556/02ae_raw_signal_targeted_variants.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163855/01f_raw_intrinsic_prompt_pack_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165430/01g_raw_template_control_pack_report.json` | `partial` | Sprint 241 introduced impact profiles/gates; 242-243 remediation loops; 244 first-pass autofill; 245 intrinsic constraints only (no uplift); 246 multishot closure; 247 single-shot shaping closure; 248 raw candidate search no uplift; 249 no-uplift guardrail; 250 closure ladder; 251 richer raw variants no uplift; 252 history-aware ladder routing; 253 batch ladder analytics; 254 raw gap backlog synthesis from `raw_only` attempts identifies top missing raw signals; 255 adds first-pass raw top-gap hardening that closes hard-sample profile coverage (`failing_profile_count 7 -> 0`) by injecting missing ops/contracts/reasons and minimum task depth; 256 adds top-gap weighted candidate selection telemetry and tie-break policy; 257 adds hard fail policy when top-gap uplift is required but absent (`exit 18`); 258 injects backlog-driven top-gap normalized requirements before raw generation; 259 adds adaptive retry with expanded top-gap requirements and deterministic retry scoring; 260 adds signal-targeted variant expansion keyed to missing-signal classes and increases intrinsic search breadth (`available_variants 2 -> 8`); 261 adds intrinsic prompt-pack constraint injection keyed to gap signals; 262 adds explicit output-shape template-control pack injection (schema + example + signal mapping) pre-generation. Measured intrinsic output still shows no uplift on this sample (`best_top_gap_score=0`, `failing_profile_count=1`). | Add deterministic post-generation structural projector for raw candidates before scoring (split/normalize schema to enforce missing ops/contracts/reason keywords without full hardening) |
| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_20260226_151651/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_enforce_20260226_151709/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint245_intrinsic_20260226_151835/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_20260226_153033/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_enforce_20260226_153045/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_20260226_153230/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_enforce_20260226_153241/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint248_rawsearch_20260226_153903/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint249_rawguard_20260226_154028/02ae_raw_candidate_search.json`, `logs/taskitem_runs/TEST_ONLY_sprint250_closure_ladder_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint253_closure_ladder_batch_20260226/closure_ladder_batch_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint254_closure_ladder_batch_20260226_r2/raw_gap_backlog.json`, `logs/taskitem_runs/TEST_ONLY_sprint255_rawharden_20260226_161924/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162423/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162633/02ae_raw_candidate_search.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/01e_raw_top_gap_requirements_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163148/02af_raw_adaptive_retry_score.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163556/02ae_raw_signal_targeted_variants.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163855/01f_raw_intrinsic_prompt_pack_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165430/01g_raw_template_control_pack_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/02ae_candidate_0_projected_report.json` | `partial` | Sprint 241 introduced impact profiles/gates; 242-243 remediation loops; 244 first-pass autofill; 245 intrinsic constraints only (no uplift); 246 multishot closure; 247 single-shot shaping closure; 248 raw candidate search no uplift; 249 no-uplift guardrail; 250 closure ladder; 251 richer raw variants no uplift; 252 history-aware ladder routing; 253 batch ladder analytics; 254 raw gap backlog synthesis from `raw_only` attempts identifies top missing raw signals; 255 adds first-pass raw top-gap hardening that closes hard-sample profile coverage (`failing_profile_count 7 -> 0`) by injecting missing ops/contracts/reasons and minimum task depth; 256 adds top-gap weighted candidate selection telemetry and tie-break policy; 257 adds hard fail policy when top-gap uplift is required but absent (`exit 18`); 258 injects backlog-driven top-gap normalized requirements before raw generation; 259 adds adaptive retry with expanded top-gap requirements and deterministic retry scoring; 260 adds signal-targeted variant expansion; 261 adds intrinsic prompt-pack constraints; 262 adds output-shape template-control pack; 263 adds deterministic structural projector before scoring. On hard sample, raw-score telemetry now uplifts (`best_failing_profile_count 1 -> 0`, `best_top_gap_score 0 -> 88`) but final `native_impact_coverage` still reports `1`, exposing scorer/gate parity mismatch. | Align score-native profile checker semantics with native_impact_coverage checker and gate on parity consistency |
## What Was Covered Today (Sprints 175-184)

18
docs/progress_log_2026-02-26.md
View File

@@ -524,3 +524,21 @@
- Current measured result on hard sample:
- template control injected (`requirement_count=3`, `selected_signal_count=8`)
- intrinsic outcome unchanged (`selected_variant=0`, `best_top_gap_score=0`, `failing_profile_count=1`).
## Sprint 263 Added (Same Day)
- Added deterministic raw structural projector:
- `tools/mcp/project_raw_candidate_structure.py`
- pipeline control:
- `WSTONE_NATIVE_RAW_STRUCTURAL_PROJECTOR`
- summary packet:
- `native_raw_structural_projector`
- Dated A/B artifacts:
- OFF: `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165737/00_summary.json`
- ON: `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/00_summary.json`
- Current measured result on hard sample:
- raw search score uplift under projector:
- `best_failing_profile_count 1 -> 0`
- `best_top_gap_score 0 -> 88`
- `best_task_count 2 -> 6`
- but `native_impact_coverage.failing_profile_count` remained `1`, exposing scorer/gate parity mismatch.

46
docs/sprint263_execution_tracker_2026-02-26.md Normal file
View File

@@ -0,0 +1,46 @@
# Sprint 263 Execution Tracker - 2026-02-26
## Scope
- `sprint263_plan.md`
## Implemented
New tool:
- `tools/mcp/project_raw_candidate_structure.py`
- Normalizes raw tasks against top-gap missing-signal classes.
- Fills missing ops/reasons/execution contract fields.
- Expands shallow task sets to gap-indicated minimum count.
Pipeline integration in `tools/mcp/run_sprint_taskitem_pipeline.sh`:
- Added:
- `WSTONE_NATIVE_RAW_STRUCTURAL_PROJECTOR`
- Raw candidate search now optionally projects:
- baseline candidate (`02ae_candidate_0_projected_tasks.json`)
- each generated candidate variant (`02ae_candidate_<n>_projected_tasks.json`)
- adaptive retry candidate when enabled
- Added summary packet:
- `native_raw_structural_projector`
## Validation Artifacts
OFF:
- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165737/00_summary.json`
ON:
- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/00_summary.json`
- projected artifacts:
- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/02ae_candidate_0_projected_report.json`
- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/02ae_candidate_0_projected_tasks.json`
Observed result on hard sample:
- Raw search score improved sharply under projector:
- `best_failing_profile_count: 1 -> 0`
- `best_top_gap_score: 0 -> 88`
- `best_task_count: 2 -> 6`
- Residual parity gap:
- `native_impact_coverage.failing_profile_count` remained `1`
- indicates score-tool vs coverage-gate semantics are not yet aligned.
## Explicit Completion Signal
- Sprint 263: `DONE` (implemented + validated; uncovered scorer/gate parity gap)