Add backlog-driven raw top-gap requirement injection (sprint 258)

docs/constructive_editing_runtime_plan.md

@@ -223,3 +223,8 @@ Planning runtime controls (active):
 - raw candidate search can enforce top-gap uplift as hard policy:
   - `WSTONE_NATIVE_RAW_TOP_GAP_REQUIRE_UPLIFT`
   - failure code `18` when no weighted uplift is achieved
+- raw candidate generation can ingest backlog-driven top-gap normalized requirements:
+  - `tools/mcp/synthesize_raw_top_gap_requirements.py`
+  - `WSTONE_NATIVE_RAW_TOP_GAP_REQUIREMENTS`
+  - `WSTONE_NATIVE_RAW_TOP_GAP_MAX_SIGNALS`
+  - `native_raw_top_gap_requirements`

docs/generator_readiness_gap_registry_2026-02-26.md

@@ -47,7 +47,7 @@ This is the canonical dated registry for "not production-ready" generator gaps.
 | GR-018 | Performance/security/rollout constrained refactor enforcement gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/results.jsonl` | `partial` | Hard checks now enforce migration rollback + data-loss policy, security deny-by-default, SLO p95 presence, and rollout staged+abort policy. Enforcement is contract-level; generator capability under these constraints is still weak in C++ AB path. | Constraint-aware generation follow-up |
 | GR-019 | Parity-blocked readiness load (gating without capability closure) | `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r7/summary.json`, `logs/taskitem_runs/challenging_subset_prod_20260226_r7/summary.json`, `docs/sprint225_227_execution_tracker_2026-02-26.md` | `partial` | Sprint 225-227 reduced blocked parity load from `6` to `0` on both tracked hard catalogs while keeping unresolved divergence at `0`. This closes immediate safety debt for current corpora, but robustness is still contingent on pattern-driven repair classes. | Generalize repairs beyond queue-shaped transpile outputs |
 | GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`, `logs/taskitem_runs/TEST_ONLY_sprint239_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145631/00_summary.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md`, `docs/sprint238_execution_tracker_2026-02-26.md`, `docs/sprint239_execution_tracker_2026-02-26.md`, `docs/sprint240_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Sprint 238 added native decomposition hard gate in pipeline (`min task count`, `min semantic signal count`) so weak native generation can be blocked before fallback masking. Sprint 239 added native reason enrichment and improved semantic signal density (`0 -> 6`). Sprint 240 added native decomposition retry with explicit minimum-task policy; on current sample retry attempted but did not improve depth (`2 -> 2`). | Native decomposition task-depth upgrade (increase native task granularity beyond 2) |
-| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_20260226_151651/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_enforce_20260226_151709/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint245_intrinsic_20260226_151835/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_20260226_153033/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_enforce_20260226_153045/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_20260226_153230/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_enforce_20260226_153241/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint248_rawsearch_20260226_153903/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint249_rawguard_20260226_154028/02ae_raw_candidate_search.json`, `logs/taskitem_runs/TEST_ONLY_sprint250_closure_ladder_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint253_closure_ladder_batch_20260226/closure_ladder_batch_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint254_closure_ladder_batch_20260226_r2/raw_gap_backlog.json`, `logs/taskitem_runs/TEST_ONLY_sprint255_rawharden_20260226_161924/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162423/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162633/02ae_raw_candidate_search.json` | `partial` | Sprint 241 introduced impact profiles/gates; 242-243 remediation loops; 244 first-pass autofill; 245 intrinsic constraints only (no uplift); 246 multishot closure; 247 single-shot shaping closure; 248 raw candidate search no uplift; 249 no-uplift guardrail; 250 closure ladder; 251 richer raw variants no uplift; 252 history-aware ladder routing; 253 batch ladder analytics; 254 raw gap backlog synthesis from `raw_only` attempts identifies top missing raw signals; 255 adds first-pass raw top-gap hardening that closes hard-sample profile coverage (`failing_profile_count 7 -> 0`) by injecting missing ops/contracts/reasons and minimum task depth; 256 adds top-gap weighted candidate selection telemetry and tie-break policy; 257 adds hard fail policy when top-gap uplift is required but absent (`exit 18`). Residual gap remains intrinsic: raw generation still fails to produce uplift on this sample without hardening/shaping overlays. | Add intrinsic raw requirement synthesis loop that explicitly targets top weighted missing signals before raw generation retries |
+| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_20260226_151651/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_enforce_20260226_151709/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint245_intrinsic_20260226_151835/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_20260226_153033/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_enforce_20260226_153045/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_20260226_153230/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_enforce_20260226_153241/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint248_rawsearch_20260226_153903/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint249_rawguard_20260226_154028/02ae_raw_candidate_search.json`, `logs/taskitem_runs/TEST_ONLY_sprint250_closure_ladder_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint253_closure_ladder_batch_20260226/closure_ladder_batch_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint254_closure_ladder_batch_20260226_r2/raw_gap_backlog.json`, `logs/taskitem_runs/TEST_ONLY_sprint255_rawharden_20260226_161924/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162423/00_summary.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162633/02ae_raw_candidate_search.json`, `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/01e_raw_top_gap_requirements_report.json` | `partial` | Sprint 241 introduced impact profiles/gates; 242-243 remediation loops; 244 first-pass autofill; 245 intrinsic constraints only (no uplift); 246 multishot closure; 247 single-shot shaping closure; 248 raw candidate search no uplift; 249 no-uplift guardrail; 250 closure ladder; 251 richer raw variants no uplift; 252 history-aware ladder routing; 253 batch ladder analytics; 254 raw gap backlog synthesis from `raw_only` attempts identifies top missing raw signals; 255 adds first-pass raw top-gap hardening that closes hard-sample profile coverage (`failing_profile_count 7 -> 0`) by injecting missing ops/contracts/reasons and minimum task depth; 256 adds top-gap weighted candidate selection telemetry and tie-break policy; 257 adds hard fail policy when top-gap uplift is required but absent (`exit 18`); 258 injects backlog-driven top-gap normalized requirements before raw generation, but measured intrinsic output still shows no uplift on this sample (`best_top_gap_score=0`, `failing_profile_count=1`). | Add adaptive raw retry loop that iteratively re-synthesizes and expands top-gap requirements when uplift remains zero |
 
 ## What Was Covered Today (Sprints 175-184)
 

docs/progress_log_2026-02-26.md

@@ -441,3 +441,20 @@
 - Current measured result on hard sample:
   - `baseline_top_gap_score=0`, `best_top_gap_score=0`
   - run blocked correctly when uplift requirement is enabled.
+
+## Sprint 258 Added (Same Day)
+
+- Added backlog-driven raw requirement injection:
+  - `tools/mcp/synthesize_raw_top_gap_requirements.py`
+  - pipeline controls:
+    - `WSTONE_NATIVE_RAW_TOP_GAP_REQUIREMENTS`
+    - `WSTONE_NATIVE_RAW_TOP_GAP_MAX_SIGNALS`
+  - summary packet:
+    - `native_raw_top_gap_requirements`
+- Dated A/B artifacts (requirements OFF/ON):
+  - `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162853/00_summary.json`
+  - `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/00_summary.json`
+  - `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/01e_raw_top_gap_requirements_report.json`
+- Current measured result on hard sample:
+  - injection selected `5` top signals from `20`
+  - intrinsic raw outcome unchanged (`selected_variant=0`, `best_top_gap_score=0`, `failing_profile_count=1`).

docs/sprint258_execution_tracker_2026-02-26.md

@@ -0,0 +1,41 @@
+# Sprint 258 Execution Tracker - 2026-02-26
+
+## Scope
+- `sprint258_plan.md`
+
+## Implemented
+
+New tool:
+- `tools/mcp/synthesize_raw_top_gap_requirements.py`
+  - Reads `raw_gap_backlog.json`.
+  - Selects top `N` missing signals.
+  - Emits normalized requirement objects suitable for `normalizedRequirements`.
+
+Pipeline integration in `tools/mcp/run_sprint_taskitem_pipeline.sh`:
+- Added:
+  - `WSTONE_NATIVE_RAW_TOP_GAP_REQUIREMENTS`
+  - `WSTONE_NATIVE_RAW_TOP_GAP_MAX_SIGNALS`
+- Added artifacts:
+  - `01e_raw_top_gap_requirements.json`
+  - `01e_raw_top_gap_requirements_report.json`
+- Added summary packet:
+  - `native_raw_top_gap_requirements`
+
+## Validation Artifacts
+
+OFF:
+- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162853/00_summary.json`
+
+ON:
+- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/00_summary.json`
+- `logs/taskitem_runs/01a_fallback_intake_spec_20260226_162924/01e_raw_top_gap_requirements_report.json`
+
+Observed result on hard sample:
+- Injection enabled with `selected_signal_count=5` from `input_signal_count=20`.
+- `selected_variant` remained `0`.
+- `best_top_gap_score` remained `0`.
+- `failing_profile_count` remained `1`.
+
+## Explicit Completion Signal
+
+- Sprint 258: `DONE` (implemented + validated; intrinsic uplift still pending)

editor/src/Sprint258IntegrationSummary.h

@@ -0,0 +1,6 @@
+#pragma once
+
+// Sprint 258 integration summary:
+// - Added backlog-driven raw top-gap requirement synthesis tool.
+// - Pipeline can inject top weighted missing-signal requirements before raw candidate generation.
+// - Summary now reports injected top-gap requirement metadata for intrinsic tuning loops.

sprint258_plan.md

@@ -0,0 +1,11 @@
+# Sprint 258 Plan: Backlog-Driven Raw Requirement Injection
+
+## Goal
+Translate prioritized raw-gap backlog signals into concrete normalized requirements and inject them before raw candidate generation to improve intrinsic first-pass guidance.
+
+## Steps
+- Step 2337: Add synthesis tool mapping backlog missing-signal classes to normalized requirement clauses.
+- Step 2338: Add pipeline controls for enabling top-gap requirement injection and max signal count.
+- Step 2339: Emit raw top-gap requirement injection telemetry in summary packet.
+- Step 2340: Validate OFF/ON behavior against the hard sample.
+- Step 2341: Add `Sprint258IntegrationSummary.h` and execution tracker.

tools/mcp/run_sprint_taskitem_pipeline.sh

@@ -56,6 +56,8 @@ NATIVE_RAW_HARDEN_TOP_GAPS="${WSTONE_NATIVE_RAW_HARDEN_TOP_GAPS:-0}"
 NATIVE_RAW_TOP_GAP_WEIGHTED_SELECT="${WSTONE_NATIVE_RAW_TOP_GAP_WEIGHTED_SELECT:-0}"
 NATIVE_RAW_TOP_GAP_BACKLOG_FILE="${WSTONE_NATIVE_RAW_TOP_GAP_BACKLOG_FILE:-}"
 NATIVE_RAW_TOP_GAP_REQUIRE_UPLIFT="${WSTONE_NATIVE_RAW_TOP_GAP_REQUIRE_UPLIFT:-0}"
+NATIVE_RAW_TOP_GAP_REQUIREMENTS="${WSTONE_NATIVE_RAW_TOP_GAP_REQUIREMENTS:-0}"
+NATIVE_RAW_TOP_GAP_MAX_SIGNALS="${WSTONE_NATIVE_RAW_TOP_GAP_MAX_SIGNALS:-5}"
 EXTRA_NORMALIZED_REQUIREMENTS_FILE="${WSTONE_EXTRA_NORMALIZED_REQUIREMENTS_FILE:-}"
 EXTRA_TASKS_FILE="${WSTONE_EXTRA_TASKS_FILE:-}"
 CAPABILITY_SIGNALS_JSON="${WSTONE_CAPABILITY_SIGNALS_JSON:-}"
@@ -119,6 +121,7 @@ NATIVE_SINGLESHOT_PROFILE_SHAPE_JSON='{}'
 NATIVE_MULTISHOT_JSON='{}'
 NATIVE_RAW_CANDIDATE_SEARCH_JSON='{}'
 NATIVE_RAW_HARDENING_JSON='{}'
+NATIVE_RAW_TOP_GAP_REQUIREMENTS_JSON='{}'
 INTRINSIC_REQS_JSON='[]'
 EXTRA_NORMALIZED_REQUIREMENTS_JSON='[]'
 EXTRA_TASKS_JSON='[]'
@@ -334,6 +337,22 @@ if [[ "$NATIVE_INTRINSIC_BOOST" == "1" ]]; then
 else
   NATIVE_INTRINSIC_BOOST_JSON='{"enabled":false,"requirement_count":0}'
 fi
+if [[ "$NATIVE_RAW_TOP_GAP_REQUIREMENTS" == "1" && -n "$NATIVE_RAW_TOP_GAP_BACKLOG_FILE" && -f "$NATIVE_RAW_TOP_GAP_BACKLOG_FILE" ]]; then
+  python3 "$ROOT_DIR/tools/mcp/synthesize_raw_top_gap_requirements.py" \
+    --top-gaps "$NATIVE_RAW_TOP_GAP_BACKLOG_FILE" \
+    --max-signals "$NATIVE_RAW_TOP_GAP_MAX_SIGNALS" \
+    --out "$OUT_DIR/01e_raw_top_gap_requirements.json" \
+    --out-report "$OUT_DIR/01e_raw_top_gap_requirements_report.json" >/dev/null
+  RAW_TOP_GAP_REQS_JSON="$(cat "$OUT_DIR/01e_raw_top_gap_requirements.json")"
+  NORMALIZED_REQS="$(jq -nc --argjson base "$NORMALIZED_REQS" --argjson extra "$RAW_TOP_GAP_REQS_JSON" '$base + $extra')"
+  NATIVE_RAW_TOP_GAP_REQUIREMENTS_JSON="$(jq -nc \
+    --argjson enabled true \
+    --arg backlog_file "$NATIVE_RAW_TOP_GAP_BACKLOG_FILE" \
+    --argjson report "$(cat "$OUT_DIR/01e_raw_top_gap_requirements_report.json")" \
+    '{enabled:$enabled, backlog_file:$backlog_file, report:$report}')"
+else
+  NATIVE_RAW_TOP_GAP_REQUIREMENTS_JSON='{"enabled":false}'
+fi
 CONFLICTS="$(printf '%s' "$INTAKE_JSON" | jq '.conflicts // []')"
 GEN_ARGS="$(jq -nc --argjson nr "$NORMALIZED_REQS" --argjson cf "$CONFLICTS" --arg strict "$STRICT_EXECUTION_CONTRACT" \
   '{normalizedRequirements:$nr,conflicts:$cf,strictExecutionContract:($strict == "1")}')"
@@ -856,6 +875,7 @@ SUMMARY_JSON="$(jq -nc \
   --argjson native_decomposition_retry "$NATIVE_DECOMP_RETRY_JSON" \
   --argjson native_profile_autofill "$NATIVE_PROFILE_AUTOFILL_JSON" \
   --argjson native_intrinsic_boost "$NATIVE_INTRINSIC_BOOST_JSON" \
+  --argjson native_raw_top_gap_requirements "$NATIVE_RAW_TOP_GAP_REQUIREMENTS_JSON" \
   --argjson native_raw_candidate_search "$NATIVE_RAW_CANDIDATE_SEARCH_JSON" \
   --argjson native_raw_hardening "$NATIVE_RAW_HARDENING_JSON" \
   --argjson native_single_shot_profile_shape "$NATIVE_SINGLESHOT_PROFILE_SHAPE_JSON" \
@@ -885,6 +905,7 @@ SUMMARY_JSON="$(jq -nc \
     native_decomposition_retry: $native_decomposition_retry,
     native_profile_autofill: $native_profile_autofill,
     native_intrinsic_boost: $native_intrinsic_boost,
+    native_raw_top_gap_requirements: $native_raw_top_gap_requirements,
     native_raw_candidate_search: $native_raw_candidate_search,
     native_raw_hardening: $native_raw_hardening,
     native_single_shot_profile_shape: $native_single_shot_profile_shape,

tools/mcp/synthesize_raw_top_gap_requirements.py

@@ -0,0 +1,91 @@
+#!/usr/bin/env python3
+import argparse
+import json
+from pathlib import Path
+from typing import Dict, List
+
+
+def load_json(path: Path):
+    with path.open("r", encoding="utf-8") as f:
+        return json.load(f)
+
+
+def make_requirement(signal: str) -> str:
+    if signal.startswith("missing_prerequisite_op:"):
+        op = signal.split(":", 1)[1]
+        return f"Include prerequisite operation `{op}` in each relevant task."
+    if signal.startswith("missing_execution_contract:"):
+        field = signal.split(":", 1)[1]
+        return f"Set executionContract.{field}=true for deterministic-safe tasks."
+    if signal.startswith("missing_reason_keyword:"):
+        key = signal.split(":", 1)[1]
+        return f"Include reason text covering `{key}` risk/contract intent."
+    if signal.startswith("native_task_count<"):
+        target = signal.split("<", 1)[1]
+        return f"Produce at least {target} native taskitems for required impact coverage."
+    return f"Address missing signal: {signal}"
+
+
+def main() -> int:
+    p = argparse.ArgumentParser(description="Synthesize normalized requirements from prioritized raw top-gap signals.")
+    p.add_argument("--top-gaps", required=True)
+    p.add_argument("--max-signals", type=int, default=5)
+    p.add_argument("--out", required=True)
+    p.add_argument("--out-report", required=True)
+    args = p.parse_args()
+
+    backlog = load_json(Path(args.top_gaps))
+    prioritized = list(backlog.get("prioritized_missing") or [])
+    selected = prioritized[: max(0, args.max_signals)]
+
+    reqs: List[Dict] = []
+    selected_signals: List[Dict] = []
+    for row in selected:
+        signal = str(row.get("missing", ""))
+        if not signal:
+            continue
+        reqs.append(
+            {
+                "requirementId": f"raw-top-gap-{len(reqs)+1}",
+                "kind": "constraint",
+                "normalizedText": make_requirement(signal),
+                "anchor": "raw_top_gap_backlog",
+                "sourceLine": 0,
+                "ambiguous": False,
+            }
+        )
+        selected_signals.append(
+            {
+                "missing": signal,
+                "count": int(row.get("count", 0) or 0),
+            }
+        )
+
+    with Path(args.out).open("w", encoding="utf-8") as f:
+        json.dump(reqs, f, indent=2, sort_keys=True)
+        f.write("\n")
+    report = {
+        "status": "ok",
+        "input_signal_count": len(prioritized),
+        "selected_signal_count": len(selected_signals),
+        "selected_signals": selected_signals,
+    }
+    with Path(args.out_report).open("w", encoding="utf-8") as f:
+        json.dump(report, f, indent=2, sort_keys=True)
+        f.write("\n")
+
+    print(
+        json.dumps(
+            {
+                "status": "ok",
+                "selected_signal_count": len(selected_signals),
+                "out": args.out,
+            },
+            sort_keys=True,
+        )
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())