diff --git a/docs/constructive_editing_runtime_plan.md b/docs/constructive_editing_runtime_plan.md index 7311882..2a67dd2 100644 --- a/docs/constructive_editing_runtime_plan.md +++ b/docs/constructive_editing_runtime_plan.md @@ -251,3 +251,5 @@ Planning runtime controls (active): - `tools/mcp/project_raw_candidate_structure.py` - `WSTONE_NATIVE_RAW_STRUCTURAL_PROJECTOR` - `native_raw_structural_projector` +- raw candidate selection now persists selected tasks into generation artifacts for gate parity: + - keeps `native_raw_candidate_search` and `native_impact_coverage` on the same task set diff --git a/docs/generator_readiness_gap_registry_2026-02-26.md b/docs/generator_readiness_gap_registry_2026-02-26.md index 2679494..aa006de 100644 --- a/docs/generator_readiness_gap_registry_2026-02-26.md +++ b/docs/generator_readiness_gap_registry_2026-02-26.md @@ -47,7 +47,7 @@ This is the canonical dated registry for "not production-ready" generator gaps. | GR-018 | Performance/security/rollout constrained refactor enforcement gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/results.jsonl` | `partial` | Hard checks now enforce migration rollback + data-loss policy, security deny-by-default, SLO p95 presence, and rollout staged+abort policy. Enforcement is contract-level; generator capability under these constraints is still weak in C++ AB path. | Constraint-aware generation follow-up | | GR-019 | Parity-blocked readiness load (gating without capability closure) | `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r7/summary.json`, `logs/taskitem_runs/challenging_subset_prod_20260226_r7/summary.json`, `docs/sprint225_227_execution_tracker_2026-02-26.md` | `partial` | Sprint 225-227 reduced blocked parity load from `6` to `0` on both tracked hard catalogs while keeping unresolved divergence at `0`. This closes immediate safety debt for current corpora, but robustness is still contingent on pattern-driven repair classes. | Generalize repairs beyond queue-shaped transpile outputs | | GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`, `logs/taskitem_runs/TEST_ONLY_sprint239_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145631/00_summary.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md`, `docs/sprint238_execution_tracker_2026-02-26.md`, `docs/sprint239_execution_tracker_2026-02-26.md`, `docs/sprint240_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Sprint 238 added native decomposition hard gate in pipeline (`min task count`, `min semantic signal count`) so weak native generation can be blocked before fallback masking. Sprint 239 added native reason enrichment and improved semantic signal density (`0 -> 6`). Sprint 240 added native decomposition retry with explicit minimum-task policy; on current sample retry attempted but did not improve depth (`2 -> 2`). | Native decomposition task-depth upgrade (increase native task granularity beyond 2) | -| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_20260226_151651/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_enforce_20260226_151709/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint245_intrinsic_20260226_151835/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_20260226_153033/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_enforce_20260226_153045/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_20260226_153230/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_enforce_20260226_153241/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint248_rawsearch_20260226_153903/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint249_rawguard_20260226_154028/02ae_raw_candidate_search.json`, `logs/taskitem_runs/TEST_ONLY_sprint250_closure_ladder_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint253_closure_ladder_batch_20260226/closure_ladder_batch_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint254_closure_ladder_batch_20260226_r2/raw_gap_backlog.json`, `logs/taskitem_runs/TEST_ONLY_sprint255_rawharden_20260226_161924/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162423/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162633/02ae_raw_candidate_search.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/01e_raw_top_gap_requirements_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163148/02af_raw_adaptive_retry_score.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163556/02ae_raw_signal_targeted_variants.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163855/01f_raw_intrinsic_prompt_pack_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165430/01g_raw_template_control_pack_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/02ae_candidate_0_projected_report.json` | `partial` | Sprint 241 introduced impact profiles/gates; 242-243 remediation loops; 244 first-pass autofill; 245 intrinsic constraints only (no uplift); 246 multishot closure; 247 single-shot shaping closure; 248 raw candidate search no uplift; 249 no-uplift guardrail; 250 closure ladder; 251 richer raw variants no uplift; 252 history-aware ladder routing; 253 batch ladder analytics; 254 raw gap backlog synthesis from `raw_only` attempts identifies top missing raw signals; 255 adds first-pass raw top-gap hardening that closes hard-sample profile coverage (`failing_profile_count 7 -> 0`) by injecting missing ops/contracts/reasons and minimum task depth; 256 adds top-gap weighted candidate selection telemetry and tie-break policy; 257 adds hard fail policy when top-gap uplift is required but absent (`exit 18`); 258 injects backlog-driven top-gap normalized requirements before raw generation; 259 adds adaptive retry with expanded top-gap requirements and deterministic retry scoring; 260 adds signal-targeted variant expansion; 261 adds intrinsic prompt-pack constraints; 262 adds output-shape template-control pack; 263 adds deterministic structural projector before scoring. On hard sample, raw-score telemetry now uplifts (`best_failing_profile_count 1 -> 0`, `best_top_gap_score 0 -> 88`) but final `native_impact_coverage` still reports `1`, exposing scorer/gate parity mismatch. | Align score-native profile checker semantics with native_impact_coverage checker and gate on parity consistency | +| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_20260226_151651/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_enforce_20260226_151709/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint245_intrinsic_20260226_151835/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_20260226_153033/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_enforce_20260226_153045/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_20260226_153230/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_enforce_20260226_153241/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint248_rawsearch_20260226_153903/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint249_rawguard_20260226_154028/02ae_raw_candidate_search.json`, `logs/taskitem_runs/TEST_ONLY_sprint250_closure_ladder_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint253_closure_ladder_batch_20260226/closure_ladder_batch_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint254_closure_ladder_batch_20260226_r2/raw_gap_backlog.json`, `logs/taskitem_runs/TEST_ONLY_sprint255_rawharden_20260226_161924/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162423/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162633/02ae_raw_candidate_search.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/01e_raw_top_gap_requirements_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163148/02af_raw_adaptive_retry_score.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163556/02ae_raw_signal_targeted_variants.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_163855/01f_raw_intrinsic_prompt_pack_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165430/01g_raw_template_control_pack_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/02ae_candidate_0_projected_report.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_170014/06_native_impact_coverage.json` | `partial` | Sprint 241-262 established profile gating plus multiple intrinsic guidance paths; sprint 263 added deterministic structural projection before scoring and showed uplift in score telemetry; sprint 264 aligned score/gate parity by persisting selected raw candidate tasks into generation artifacts. On the hard sample with projector ON, both score and gate now pass consistently (`best_failing_profile_count=0`, `native_impact_coverage.failing_profile_count=0`, task_count `6`). Residual risk is breadth: closure is validated on this hard sample path, not yet proven across broader hard catalogs. | Batch-validate structural projector + parity path across challenging catalogs and add hard fail when score/gate diverge | ## What Was Covered Today (Sprints 175-184) diff --git a/docs/progress_log_2026-02-26.md b/docs/progress_log_2026-02-26.md index 457ac17..fef706c 100644 --- a/docs/progress_log_2026-02-26.md +++ b/docs/progress_log_2026-02-26.md @@ -542,3 +542,17 @@ - `best_top_gap_score 0 -> 88` - `best_task_count 2 -> 6` - but `native_impact_coverage.failing_profile_count` remained `1`, exposing scorer/gate parity mismatch. + +## Sprint 264 Added (Same Day) + +- Fixed scorer/gate parity in raw candidate path: + - selected raw `TASKS` are now persisted into `02_generate_taskitems.json` + - downstream impact coverage checker reads the same selected task set as raw scorer +- Dated validation artifacts: + - pre-fix mismatch reference: `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/00_summary.json` + - post-fix parity run: `logs/taskitem_runs/01a_fallback_intake_spec_20260226_170014/00_summary.json` + - post-fix coverage report: `logs/taskitem_runs/01a_fallback_intake_spec_20260226_170014/06_native_impact_coverage.json` +- Current measured result on hard sample (projector ON): + - `best_failing_profile_count=0` + - `native_impact_coverage.failing_profile_count=0` + - scorer and gate are aligned. diff --git a/docs/sprint264_execution_tracker_2026-02-26.md b/docs/sprint264_execution_tracker_2026-02-26.md new file mode 100644 index 0000000..cf0b3a0 --- /dev/null +++ b/docs/sprint264_execution_tracker_2026-02-26.md @@ -0,0 +1,32 @@ +# Sprint 264 Execution Tracker - 2026-02-26 + +## Scope +- `sprint264_plan.md` + +## Implemented + +Pipeline parity fix in `tools/mcp/run_sprint_taskitem_pipeline.sh`: +- After raw candidate selection, pipeline now persists selected `TASKS` into: + - `02_generate_taskitems.json` +- This ensures downstream `check_native_decomposition_impact_coverage.py` and summary gates evaluate the same selected task set as raw scoring. + +## Validation Artifacts + +Before parity fix (mismatch reference): +- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_165746/00_summary.json` + - raw score pass, coverage gate fail + +After parity fix: +- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_170014/00_summary.json` +- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_170014/06_native_impact_coverage.json` +- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_170014/02ae_candidate_0_score.json` + +Observed result on hard sample (projector ON): +- `native_raw_candidate_search.best_failing_profile_count=0` +- `native_impact_coverage.failing_profile_count=0` +- `taskitems.task_count=6`, `effective_task_count=6` +- scorer and gate now consistent. + +## Explicit Completion Signal + +- Sprint 264: `DONE` (implemented + validated parity closure on hard sample) diff --git a/editor/src/Sprint264IntegrationSummary.h b/editor/src/Sprint264IntegrationSummary.h new file mode 100644 index 0000000..7210447 --- /dev/null +++ b/editor/src/Sprint264IntegrationSummary.h @@ -0,0 +1,6 @@ +#pragma once + +// Sprint 264 integration summary: +// - Aligned raw candidate score path with downstream impact coverage by persisting selected tasks into generation artifacts. +// - Removed scorer/gate parity mismatch observed in sprint 263. +// - Structural projector path now yields consistent pass signals across score and gate on the hard sample. diff --git a/sprint264_plan.md b/sprint264_plan.md new file mode 100644 index 0000000..8851168 --- /dev/null +++ b/sprint264_plan.md @@ -0,0 +1,10 @@ +# Sprint 264 Plan: Raw Score/Gate Parity Alignment + +## Goal +Eliminate scorer-vs-gate semantic drift by ensuring downstream coverage checks evaluate the same selected raw candidate task set used by raw scoring. + +## Steps +- Step 2368: Persist selected raw candidate tasks back into generation artifacts. +- Step 2369: Re-run hard sample with structural projector to verify score/gate parity. +- Step 2370: Confirm `native_raw_candidate_search` and `native_impact_coverage` agree. +- Step 2371: Add `Sprint264IntegrationSummary.h` and execution tracker. diff --git a/tools/mcp/run_sprint_taskitem_pipeline.sh b/tools/mcp/run_sprint_taskitem_pipeline.sh index e54469d..c8f8c43 100755 --- a/tools/mcp/run_sprint_taskitem_pipeline.sh +++ b/tools/mcp/run_sprint_taskitem_pipeline.sh @@ -700,6 +700,9 @@ if [[ "$NATIVE_RAW_CANDIDATE_SEARCH" == "1" ]]; then exit 18 fi fi + # Keep downstream coverage gates aligned with selected raw candidate task set. + GEN_JSON="$(jq -nc --argjson base "$GEN_JSON" --argjson tasks "$TASKS" '$base + {tasks:$tasks}')" + printf '%s\n' "$GEN_JSON" > "$OUT_DIR/02_generate_taskitems.json" else NATIVE_RAW_CANDIDATE_SEARCH_JSON='{"enabled":false}' NATIVE_RAW_SIGNAL_TARGETED_VARIANTS_JSON='{"enabled":false}'