diff --git a/docs/constructive_editing_runtime_plan.md b/docs/constructive_editing_runtime_plan.md index 2d41a19..5a91ac4 100644 --- a/docs/constructive_editing_runtime_plan.md +++ b/docs/constructive_editing_runtime_plan.md @@ -150,7 +150,7 @@ Parallel planning tranche (active): - `logs/taskitem_runs/TEST_ONLY_spec_planning_baseline_20260226/deterministic_spec_readiness.json` Planning runtime controls (active): -- `sprint232_plan.md` to `sprint250_plan.md` +- `sprint232_plan.md` to `sprint252_plan.md` - semantic bridge + intake augmentation + requirement injection + expansion gating are wired into: - `tools/mcp/run_sprint_taskitem_pipeline.sh` - current policy default is native-first semantic fallback: @@ -204,3 +204,6 @@ Planning runtime controls (active): - `tools/mcp/score_native_tasks_profile_coverage.py` - closure ladder can select the minimal passing strategy automatically: - `tools/mcp/run_native_profile_closure_ladder.sh` +- closure ladder can route around historically ineffective raw-only mode: + - `tools/mcp/analyze_raw_candidate_uplift_history.py` + - `WSTONE_CLOSURE_LADDER_SKIP_RAW_WHEN_NO_UPLIFT` diff --git a/docs/generator_readiness_gap_registry_2026-02-26.md b/docs/generator_readiness_gap_registry_2026-02-26.md index cdac8a8..adc26c2 100644 --- a/docs/generator_readiness_gap_registry_2026-02-26.md +++ b/docs/generator_readiness_gap_registry_2026-02-26.md @@ -47,7 +47,7 @@ This is the canonical dated registry for "not production-ready" generator gaps. | GR-018 | Performance/security/rollout constrained refactor enforcement gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/results.jsonl` | `partial` | Hard checks now enforce migration rollback + data-loss policy, security deny-by-default, SLO p95 presence, and rollout staged+abort policy. Enforcement is contract-level; generator capability under these constraints is still weak in C++ AB path. | Constraint-aware generation follow-up | | GR-019 | Parity-blocked readiness load (gating without capability closure) | `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r7/summary.json`, `logs/taskitem_runs/challenging_subset_prod_20260226_r7/summary.json`, `docs/sprint225_227_execution_tracker_2026-02-26.md` | `partial` | Sprint 225-227 reduced blocked parity load from `6` to `0` on both tracked hard catalogs while keeping unresolved divergence at `0`. This closes immediate safety debt for current corpora, but robustness is still contingent on pattern-driven repair classes. | Generalize repairs beyond queue-shaped transpile outputs | | GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`, `logs/taskitem_runs/TEST_ONLY_sprint239_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145631/00_summary.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md`, `docs/sprint238_execution_tracker_2026-02-26.md`, `docs/sprint239_execution_tracker_2026-02-26.md`, `docs/sprint240_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Sprint 238 added native decomposition hard gate in pipeline (`min task count`, `min semantic signal count`) so weak native generation can be blocked before fallback masking. Sprint 239 added native reason enrichment and improved semantic signal density (`0 -> 6`). Sprint 240 added native decomposition retry with explicit minimum-task policy; on current sample retry attempted but did not improve depth (`2 -> 2`). | Native decomposition task-depth upgrade (increase native task granularity beyond 2) | -| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_20260226_151651/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_enforce_20260226_151709/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint245_intrinsic_20260226_151835/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_20260226_153033/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_enforce_20260226_153045/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_20260226_153230/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_enforce_20260226_153241/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint248_rawsearch_20260226_153903/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint249_rawguard_20260226_154028/02ae_raw_candidate_search.json`, `logs/taskitem_runs/TEST_ONLY_sprint250_closure_ladder_20260226/closure_ladder_summary.json` | `partial` | Sprint 241 introduced impact profiles/gates; 242-243 remediation loops; 244 first-pass autofill; 245 intrinsic constraints only (no uplift); 246 multishot closes hard sample; 247 single-shot shaping closes with multishot+autofill disabled; 248 raw candidate search found no uplift; 249 added no-uplift guardrail; 250 added automatic closure ladder and selected `single_shot_shape` as minimal passing mode on hard sample. Residual gap remains in raw generator single-shot quality without overlays. | Improve raw single-shot generator output so profile closure does not rely on shaping/multishot/autofill overlays | +| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_20260226_151651/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint244_autofill_enforce_20260226_151709/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint245_intrinsic_20260226_151835/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_20260226_153033/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint246_multishot_enforce_20260226_153045/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_20260226_153230/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint247_singleshot_enforce_20260226_153241/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint248_rawsearch_20260226_153903/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint249_rawguard_20260226_154028/02ae_raw_candidate_search.json`, `logs/taskitem_runs/TEST_ONLY_sprint250_closure_ladder_20260226/closure_ladder_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json` | `partial` | Sprint 241 introduced impact profiles/gates; 242-243 remediation loops; 244 first-pass autofill; 245 intrinsic constraints only (no uplift); 246 multishot closes hard sample; 247 single-shot shaping closes with multishot+autofill disabled; 248 raw candidate search found no uplift; 249 added no-uplift guardrail; 250 closure ladder selected minimal passing mode; 251 richer raw variants still no raw uplift (`selected_variant=0`); 252 added history-aware ladder routing to skip raw-only when historically ineffective. Residual gap remains in raw generator single-shot quality without overlays. | Improve raw single-shot generator output so profile closure does not rely on shaping/multishot/autofill overlays | ## What Was Covered Today (Sprints 175-184) diff --git a/docs/progress_log_2026-02-26.md b/docs/progress_log_2026-02-26.md index 03b3e3d..09297e2 100644 --- a/docs/progress_log_2026-02-26.md +++ b/docs/progress_log_2026-02-26.md @@ -334,3 +334,33 @@ - Current measured result on hard fullstack sample: - selected mode: `single_shot_shape` - selected run passes impact coverage with `failing_profile_count=0` + +## Sprint 251 Added (Same Day) + +- Added richer raw-variant synthesis for raw-only candidate search: + - `tools/mcp/synthesize_raw_candidate_requirement_variants.py` + - raw search now consumes profile-bundle variants +- Dated artifact: + - `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json` +- Current measured result: + - `available_variants=8`, `attempted_variants=8` + - `selected_variant=0`, `failing_profile_count 7 -> 7` + - no raw-only uplift on this hard sample. + +## Sprint 252 Added (Same Day) + +- Added raw uplift history analyzer: + - `tools/mcp/analyze_raw_candidate_uplift_history.py` +- Added closure-ladder policy routing based on historical raw uplift: + - `tools/mcp/run_native_profile_closure_ladder.sh` + - controls: + - `WSTONE_CLOSURE_LADDER_SKIP_RAW_WHEN_NO_UPLIFT` + - `WSTONE_CLOSURE_LADDER_RAW_HISTORY_GLOB` + - `WSTONE_CLOSURE_LADDER_RAW_HISTORY_MAX_RUNS` + - `WSTONE_CLOSURE_LADDER_RAW_MIN_UPLIFT_RATE` + - `WSTONE_CLOSURE_LADDER_RAW_MIN_RECORDS` +- Dated artifacts: + - `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_20260226/closure_ladder_summary.json` + - `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json` +- Verified forced-skip behavior: + - attempted modes excluded `raw_only` and started at `single_shot_shape`. diff --git a/docs/sprint251_execution_tracker_2026-02-26.md b/docs/sprint251_execution_tracker_2026-02-26.md new file mode 100644 index 0000000..c0467a7 --- /dev/null +++ b/docs/sprint251_execution_tracker_2026-02-26.md @@ -0,0 +1,29 @@ +# Sprint 251 Execution Tracker - 2026-02-26 + +## Scope +- `sprint251_plan.md` + +## Implemented + +New tool: +- `tools/mcp/synthesize_raw_candidate_requirement_variants.py` + - Builds profile-aware raw requirement bundles for candidate search. + +Pipeline integration in `tools/mcp/run_sprint_taskitem_pipeline.sh`: +- Raw candidate search now consumes synthesized variant bundles. +- Added telemetry field: + - `native_raw_candidate_search.available_variants` + +## Validation Artifact + +- `logs/taskitem_runs/TEST_ONLY_sprint251_rawvariants_20260226_154355/00_summary.json` + +Observed result on hard fullstack sample: +- `available_variants=8` +- `attempted_variants=8` +- selected variant remains baseline (`selected_variant=0`) +- `failing_profile_count` unchanged at `7` + +## Explicit Completion Signal + +- Sprint 251: `DONE` (implemented + measured; no raw-only uplift) diff --git a/docs/sprint252_execution_tracker_2026-02-26.md b/docs/sprint252_execution_tracker_2026-02-26.md new file mode 100644 index 0000000..446301a --- /dev/null +++ b/docs/sprint252_execution_tracker_2026-02-26.md @@ -0,0 +1,36 @@ +# Sprint 252 Execution Tracker - 2026-02-26 + +## Scope +- `sprint252_plan.md` + +## Implemented + +New analyzer: +- `tools/mcp/analyze_raw_candidate_uplift_history.py` + - Computes recent raw-search uplift rate from run summaries. + +Closure ladder policy routing in `tools/mcp/run_native_profile_closure_ladder.sh`: +- Added controls: + - `WSTONE_CLOSURE_LADDER_SKIP_RAW_WHEN_NO_UPLIFT` (default `1`) + - `WSTONE_CLOSURE_LADDER_RAW_HISTORY_GLOB` + - `WSTONE_CLOSURE_LADDER_RAW_HISTORY_MAX_RUNS` + - `WSTONE_CLOSURE_LADDER_RAW_MIN_UPLIFT_RATE` + - `WSTONE_CLOSURE_LADDER_RAW_MIN_RECORDS` (default `3`) +- Emits policy artifact when enabled: + - `raw_uplift_history.json` +- Dynamically removes `raw_only` from ladder modes when uplift evidence is below threshold. + +## Validation Artifacts + +- policy-enabled run: + - `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_20260226/closure_ladder_summary.json` +- forced-skip run (demonstrates raw mode removed): + - `logs/taskitem_runs/TEST_ONLY_sprint252_closure_ladder_policy_skip2_20260226/closure_ladder_summary.json` + +Observed forced-skip signal: +- `attempted_modes=[single_shot_shape,multishot,autofill]` +- selected mode remained `single_shot_shape`. + +## Explicit Completion Signal + +- Sprint 252: `DONE` (implemented + policy skip behavior verified) diff --git a/editor/src/Sprint251IntegrationSummary.h b/editor/src/Sprint251IntegrationSummary.h new file mode 100644 index 0000000..575a793 --- /dev/null +++ b/editor/src/Sprint251IntegrationSummary.h @@ -0,0 +1,6 @@ +#pragma once + +// Sprint 251 integration summary: +// - Added richer raw variant synthesis for candidate search. +// - Raw candidate search now consumes profile-bundle variants rather than single requirement slices. +// - Hard sample still shows no raw-only uplift; selected candidate remains baseline. diff --git a/editor/src/Sprint252IntegrationSummary.h b/editor/src/Sprint252IntegrationSummary.h new file mode 100644 index 0000000..a9a96c5 --- /dev/null +++ b/editor/src/Sprint252IntegrationSummary.h @@ -0,0 +1,6 @@ +#pragma once + +// Sprint 252 integration summary: +// - Added raw uplift history analyzer for closure ladder policy routing. +// - Closure ladder can skip raw-only mode when historical uplift is weak. +// - Policy behavior is configurable via threshold and minimum-record controls. diff --git a/sprint251_plan.md b/sprint251_plan.md new file mode 100644 index 0000000..26cc0e4 --- /dev/null +++ b/sprint251_plan.md @@ -0,0 +1,11 @@ +# Sprint 251 Plan: Rich Raw Variant Synthesis for Candidate Search + +## Goal +Increase raw-only search strength by synthesizing richer requirement bundles per active profile and testing if raw generator output improves without overlays. + +## Steps +- Step 2300: Add raw candidate requirement variant synthesizer. +- Step 2301: Integrate variant bundles into raw candidate search loop. +- Step 2302: Surface available-variant telemetry in summary. +- Step 2303: Re-test hard fullstack sample for raw-only uplift. +- Step 2304: Add `Sprint251IntegrationSummary.h` and execution tracker. diff --git a/sprint252_plan.md b/sprint252_plan.md new file mode 100644 index 0000000..bef106f --- /dev/null +++ b/sprint252_plan.md @@ -0,0 +1,11 @@ +# Sprint 252 Plan: Closure Ladder Policy Routing from Raw Uplift History + +## Goal +Reduce wasted raw-only attempts by making closure ladder mode selection aware of recent raw candidate uplift performance. + +## Steps +- Step 2305: Add raw uplift history analyzer. +- Step 2306: Add ladder policy controls for skip conditions. +- Step 2307: Skip `raw_only` when historical uplift rate is below threshold. +- Step 2308: Emit policy evidence artifact in ladder output directory. +- Step 2309: Add `Sprint252IntegrationSummary.h` and execution tracker. diff --git a/tools/mcp/analyze_raw_candidate_uplift_history.py b/tools/mcp/analyze_raw_candidate_uplift_history.py new file mode 100755 index 0000000..8d5aaf9 --- /dev/null +++ b/tools/mcp/analyze_raw_candidate_uplift_history.py @@ -0,0 +1,64 @@ +#!/usr/bin/env python3 +import argparse +import json +from pathlib import Path +from typing import Dict, List + + +def load_json(path: Path): + with path.open("r", encoding="utf-8") as f: + return json.load(f) + + +def main() -> int: + p = argparse.ArgumentParser(description="Analyze recent raw candidate search uplift history.") + p.add_argument("--runs-root", default="logs/taskitem_runs") + p.add_argument("--include-glob", default="*") + p.add_argument("--max-runs", type=int, default=20) + p.add_argument("--out", required=True) + args = p.parse_args() + + root = Path(args.runs_root) + summaries: List[Path] = sorted(root.glob(f"{args.include_glob}/00_summary.json"), key=lambda x: x.stat().st_mtime, reverse=True) + + rows = [] + for path in summaries[: max(0, args.max_runs)]: + try: + s = load_json(path) + except Exception: + continue + raw = s.get("native_raw_candidate_search") or {} + if not isinstance(raw, dict) or not raw.get("enabled", False): + continue + baseline = int(raw.get("baseline_failing_profile_count", 999) or 999) + best = int(raw.get("best_failing_profile_count", 999) or 999) + rows.append({ + "run_id": path.parent.name, + "baseline_failing_profile_count": baseline, + "best_failing_profile_count": best, + "uplift": int(best < baseline), + }) + + total = len(rows) + uplift_count = sum(int(r.get("uplift", 0)) for r in rows) + no_uplift_count = total - uplift_count + + out = { + "status": "ok", + "record_count": total, + "uplift_count": uplift_count, + "no_uplift_count": no_uplift_count, + "uplift_rate": round((uplift_count / total), 4) if total else 0.0, + "records": rows, + } + + with Path(args.out).open("w", encoding="utf-8") as f: + json.dump(out, f, indent=2, sort_keys=True) + f.write("\n") + + print(json.dumps({"status": "ok", "record_count": total, "uplift_rate": out["uplift_rate"], "out": args.out}, sort_keys=True)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/mcp/run_native_profile_closure_ladder.sh b/tools/mcp/run_native_profile_closure_ladder.sh index 07df4e0..187a6d5 100755 --- a/tools/mcp/run_native_profile_closure_ladder.sh +++ b/tools/mcp/run_native_profile_closure_ladder.sh @@ -14,6 +14,31 @@ if [[ ! -f "$SPEC_PATH" ]]; then fi mkdir -p "$OUT_DIR" +SKIP_RAW_WHEN_NO_UPLIFT="${WSTONE_CLOSURE_LADDER_SKIP_RAW_WHEN_NO_UPLIFT:-1}" +RAW_HISTORY_GLOB="${WSTONE_CLOSURE_LADDER_RAW_HISTORY_GLOB:-TEST_ONLY_sprint24*_raw*}" +RAW_HISTORY_MAX_RUNS="${WSTONE_CLOSURE_LADDER_RAW_HISTORY_MAX_RUNS:-20}" +RAW_MIN_UPLIFT_RATE="${WSTONE_CLOSURE_LADDER_RAW_MIN_UPLIFT_RATE:-0.05}" +RAW_MIN_RECORDS="${WSTONE_CLOSURE_LADDER_RAW_MIN_RECORDS:-3}" + +effective_modes=(raw_only single_shot_shape multishot autofill) +if [[ "$SKIP_RAW_WHEN_NO_UPLIFT" == "1" ]]; then + python3 "$ROOT_DIR/tools/mcp/analyze_raw_candidate_uplift_history.py" \ + --runs-root "$ROOT_DIR/logs/taskitem_runs" \ + --include-glob "$RAW_HISTORY_GLOB" \ + --max-runs "$RAW_HISTORY_MAX_RUNS" \ + --out "$OUT_DIR/raw_uplift_history.json" >/dev/null || true + if [[ -f "$OUT_DIR/raw_uplift_history.json" ]]; then + raw_record_count="$(jq '.record_count // 0' "$OUT_DIR/raw_uplift_history.json")" + raw_uplift_rate="$(jq '.uplift_rate // 0' "$OUT_DIR/raw_uplift_history.json")" + if [[ "$raw_record_count" -ge "$RAW_MIN_RECORDS" ]]; then + awk_cmp="$(awk -v a="$raw_uplift_rate" -v b="$RAW_MIN_UPLIFT_RATE" 'BEGIN{if (a < b) print 1; else print 0;}')" + if [[ "$awk_cmp" == "1" ]]; then + effective_modes=(single_shot_shape multishot autofill) + fi + fi + fi +fi + run_mode() { local mode="$1" local logfile="$OUT_DIR/${mode}.log" @@ -65,12 +90,11 @@ run_mode() { return "$rc" } -modes=(raw_only single_shot_shape multishot autofill) selected="" selected_run="" selected_rc=1 -for mode in "${modes[@]}"; do +for mode in "${effective_modes[@]}"; do if run_mode "$mode"; then selected="$mode" selected_run="$(cat "$OUT_DIR/${mode}.run_dir")" @@ -85,7 +109,7 @@ jq -n \ --arg selected_mode "$selected" \ --arg selected_run "$selected_run" \ --argjson selected_rc "$selected_rc" \ - --argjson attempted_modes "$(printf '%s\n' "${modes[@]}" | jq -R . | jq -s .)" \ + --argjson attempted_modes "$(printf '%s\n' "${effective_modes[@]}" | jq -R . | jq -s .)" \ '{ status: (if $selected_rc == 0 then "ok" else "fail" end), spec:$spec, diff --git a/tools/mcp/run_sprint_taskitem_pipeline.sh b/tools/mcp/run_sprint_taskitem_pipeline.sh index 360abb9..65ff8d9 100755 --- a/tools/mcp/run_sprint_taskitem_pipeline.sh +++ b/tools/mcp/run_sprint_taskitem_pipeline.sh @@ -390,13 +390,12 @@ else '{attempted:$attempted, applied:$applied, initial_task_count:$initial_task_count, target_min_task_count:$target_min_task_count}')" fi if [[ "$NATIVE_RAW_CANDIDATE_SEARCH" == "1" ]]; then - if [[ "$(printf '%s' "$INTRINSIC_REQS_JSON" | jq 'length')" -eq 0 ]]; then - python3 "$ROOT_DIR/tools/mcp/synthesize_native_intrinsic_boost_requirements.py" \ - --spec "$INPUT_FILE" \ - --profiles "$NATIVE_IMPACT_COVERAGE_PROFILES" \ - --out "$OUT_DIR/02ae_raw_candidate_requirements.json" >/dev/null - INTRINSIC_REQS_JSON="$(cat "$OUT_DIR/02ae_raw_candidate_requirements.json")" - fi + python3 "$ROOT_DIR/tools/mcp/synthesize_raw_candidate_requirement_variants.py" \ + --spec "$INPUT_FILE" \ + --profiles "$NATIVE_IMPACT_COVERAGE_PROFILES" \ + --max-variants "$NATIVE_RAW_CANDIDATE_MAX_VARIANTS" \ + --out "$OUT_DIR/02ae_raw_candidate_variants.json" >/dev/null + VARIANTS_JSON="$(cat "$OUT_DIR/02ae_raw_candidate_variants.json")" printf '%s\n' "$TASKS" > "$OUT_DIR/02ae_candidate_0_tasks.json" python3 "$ROOT_DIR/tools/mcp/score_native_tasks_profile_coverage.py" \ @@ -412,13 +411,13 @@ if [[ "$NATIVE_RAW_CANDIDATE_SEARCH" == "1" ]]; then attempted=1 successful=1 - req_total="$(printf '%s' "$INTRINSIC_REQS_JSON" | jq 'length')" + variant_total="$(printf '%s' "$VARIANTS_JSON" | jq '.variants | length')" max_variants="$NATIVE_RAW_CANDIDATE_MAX_VARIANTS" idx=0 variant=1 - while [[ "$variant" -lt "$max_variants" && "$idx" -lt "$req_total" ]]; do - req_i="$(printf '%s' "$INTRINSIC_REQS_JSON" | jq ".[$idx]")" - cand_reqs="$(jq -nc --argjson base "$NORMALIZED_REQS" --argjson req "$req_i" '$base + [$req]')" + while [[ "$variant" -lt "$max_variants" && "$idx" -lt "$variant_total" ]]; do + req_bundle="$(printf '%s' "$VARIANTS_JSON" | jq ".variants[$idx]")" + cand_reqs="$(jq -nc --argjson base "$NORMALIZED_REQS" --argjson bundle "$req_bundle" '$base + $bundle')" cand_args="$(jq -nc --argjson nr "$cand_reqs" --argjson cf "$CONFLICTS" --arg strict "$STRICT_EXECUTION_CONTRACT" \ '{normalizedRequirements:$nr,conflicts:$cf,strictExecutionContract:($strict == "1")}')" cand_raw="$(call_tool "whetstone_generate_taskitems" "$cand_args")" @@ -457,7 +456,8 @@ if [[ "$NATIVE_RAW_CANDIDATE_SEARCH" == "1" ]]; then --argjson baseline_task_count "$baseline_task_count" \ --argjson best_failing_profile_count "$best_fail" \ --argjson best_task_count "$best_task_count" \ - '{enabled:$enabled, attempted_variants:$attempted, successful_variants:$successful, selected_variant:$best_variant, baseline_failing_profile_count:$baseline_failing_profile_count, baseline_task_count:$baseline_task_count, best_failing_profile_count:$best_failing_profile_count, best_task_count:$best_task_count}')" + --argjson available_variants "$variant_total" \ + '{enabled:$enabled, attempted_variants:$attempted, successful_variants:$successful, available_variants:$available_variants, selected_variant:$best_variant, baseline_failing_profile_count:$baseline_failing_profile_count, baseline_task_count:$baseline_task_count, best_failing_profile_count:$best_failing_profile_count, best_task_count:$best_task_count}')" printf '%s\n' "$NATIVE_RAW_CANDIDATE_SEARCH_JSON" > "$OUT_DIR/02ae_raw_candidate_search.json" if [[ "$NATIVE_RAW_CANDIDATE_REQUIRE_UPLIFT" == "1" && "$best_fail" -ge "$baseline_fail" ]]; then echo "error: raw candidate search did not improve failing profile count" >&2 diff --git a/tools/mcp/synthesize_raw_candidate_requirement_variants.py b/tools/mcp/synthesize_raw_candidate_requirement_variants.py new file mode 100755 index 0000000..9a984a4 --- /dev/null +++ b/tools/mcp/synthesize_raw_candidate_requirement_variants.py @@ -0,0 +1,131 @@ +#!/usr/bin/env python3 +import argparse +import json +from pathlib import Path +from typing import Dict, List + + +def load_json(path: Path): + with path.open("r", encoding="utf-8") as f: + return json.load(f) + + +def write_json(path: Path, obj) -> None: + with path.open("w", encoding="utf-8") as f: + json.dump(obj, f, indent=2, sort_keys=True) + f.write("\n") + + +def activate_profiles(profiles: List[Dict], spec_text: str) -> List[Dict]: + s = spec_text.lower() + out = [] + for p in profiles: + keys = [str(k).lower() for k in (p.get("trigger_keywords") or [])] + if keys and any(k in s for k in keys): + out.append(p) + return out + + +def req_from_profile(p: Dict, idx: int) -> Dict: + pid = str(p.get("id", f"profile_{idx}")) + min_tasks = int(p.get("min_native_task_count", 0) or 0) + reasons = [str(x) for x in (p.get("required_reason_keywords") or [])] + ops = [str(x) for x in (p.get("required_prerequisite_ops") or [])] + contract = [str(x) for x in (p.get("required_execution_contract") or [])] + text = ( + f"raw candidate policy {pid}: produce >= {min_tasks} concrete taskitems; " + f"reasons include {', '.join(reasons) if reasons else 'none'}; " + f"ops include {', '.join(ops) if ops else 'none'}; " + f"execution contract include {', '.join(contract) if contract else 'none'}; " + "avoid generic or umbrella tasks." + ) + return { + "requirementId": f"raw-variant-{pid}", + "kind": "constraint", + "normalizedText": text, + "anchor": "raw_candidate_variant", + "sourceLine": 0, + "ambiguous": False, + } + + +def global_decomposition_req(active_count: int, min_target: int) -> Dict: + return { + "requirementId": "raw-variant-global-decomposition", + "kind": "constraint", + "normalizedText": ( + f"raw decomposition hard policy: output at least {min_target} concrete taskitems " + f"for {active_count} active impact profiles; each task must have deterministic executionContract and explicit prerequisiteOps." + ), + "anchor": "raw_candidate_variant", + "sourceLine": 0, + "ambiguous": False, + } + + +def build_variants(active: List[Dict], max_variants: int) -> List[List[Dict]]: + if max_variants <= 0: + return [] + + profile_reqs = [req_from_profile(p, i) for i, p in enumerate(active)] + min_target = max(5, len(active) + 2) + g = global_decomposition_req(len(active), min_target) + + variants: List[List[Dict]] = [] + + # Variant 1: global only + variants.append([g]) + + # Variant 2: global + all profiles + variants.append([g] + profile_reqs) + + # Variant 3+: one profile at a time + global + for r in profile_reqs: + variants.append([g, r]) + + # Variant: paired profile bundles + for i in range(0, len(profile_reqs), 2): + bundle = [g] + profile_reqs[i:i+2] + variants.append(bundle) + + # Deduplicate by requirementId tuple ordering + seen = set() + out: List[List[Dict]] = [] + for v in variants: + key = tuple(x.get("requirementId", "") for x in v) + if key in seen: + continue + seen.add(key) + out.append(v) + if len(out) >= max_variants: + break + return out + + +def main() -> int: + p = argparse.ArgumentParser(description="Synthesize raw candidate requirement variants from active profiles.") + p.add_argument("--spec", required=True) + p.add_argument("--profiles", required=True) + p.add_argument("--max-variants", type=int, default=8) + p.add_argument("--out", required=True) + args = p.parse_args() + + spec_text = Path(args.spec).read_text(encoding="utf-8", errors="ignore") + profiles_doc = load_json(Path(args.profiles)) + profiles = list(profiles_doc.get("profiles") or []) + active = activate_profiles(profiles, spec_text) + + variants = build_variants(active, args.max_variants) + payload = { + "status": "ok", + "active_profile_count": len(active), + "variant_count": len(variants), + "variants": variants, + } + write_json(Path(args.out), payload) + print(json.dumps({"status": "ok", "active_profile_count": len(active), "variant_count": len(variants), "out": args.out}, sort_keys=True)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main())