From 3c04216a54563fbef5e15fe338506c95207efbe3 Mon Sep 17 00:00:00 2001 From: Bill Date: Thu, 26 Feb 2026 15:05:21 -0700 Subject: [PATCH] Add impact-by-impact native decomposition coverage gates (sprint 241) --- docs/constructive_editing_runtime_plan.md | 7 +- ...rator_readiness_gap_registry_2026-02-26.md | 1 + ...ve_decomposition_impact_list_2026-02-26.md | 68 +++++++ docs/progress_log_2026-02-26.md | 19 ++ .../sprint241_execution_tracker_2026-02-26.md | 55 ++++++ editor/src/Sprint241IntegrationSummary.h | 6 + sprint241_plan.md | 12 ++ tools/mcp/analyze_native_impact_coverage.py | 61 +++++++ ...ck_native_decomposition_impact_coverage.py | 166 ++++++++++++++++++ .../native_decomposition_impact_profiles.json | 113 ++++++++++++ tools/mcp/run_native_impact_coverage_gate.sh | 28 +++ tools/mcp/run_sprint_taskitem_pipeline.sh | 24 +++ 12 files changed, 559 insertions(+), 1 deletion(-) create mode 100644 docs/native_decomposition_impact_list_2026-02-26.md create mode 100644 docs/sprint241_execution_tracker_2026-02-26.md create mode 100644 editor/src/Sprint241IntegrationSummary.h create mode 100644 sprint241_plan.md create mode 100755 tools/mcp/analyze_native_impact_coverage.py create mode 100755 tools/mcp/check_native_decomposition_impact_coverage.py create mode 100644 tools/mcp/profiles/native_decomposition_impact_profiles.json create mode 100755 tools/mcp/run_native_impact_coverage_gate.sh diff --git a/docs/constructive_editing_runtime_plan.md b/docs/constructive_editing_runtime_plan.md index 7be4402..45771ae 100644 --- a/docs/constructive_editing_runtime_plan.md +++ b/docs/constructive_editing_runtime_plan.md @@ -150,7 +150,7 @@ Parallel planning tranche (active): - `logs/taskitem_runs/TEST_ONLY_spec_planning_baseline_20260226/deterministic_spec_readiness.json` Planning runtime controls (active): -- `sprint232_plan.md` to `sprint240_plan.md` +- `sprint232_plan.md` to `sprint241_plan.md` - semantic bridge + intake augmentation + requirement injection + expansion gating are wired into: - `tools/mcp/run_sprint_taskitem_pipeline.sh` - current policy default is native-first semantic fallback: @@ -169,3 +169,8 @@ Planning runtime controls (active): - `WSTONE_NATIVE_DECOMP_RETRY` - `WSTONE_NATIVE_DECOMP_TARGET_MIN_TASKS` - `native_decomposition_retry` +- impact-specific native decomposition coverage can be enforced: + - `WSTONE_NATIVE_IMPACT_COVERAGE_GATE` + - `WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE` + - `WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES` + - `native_impact_coverage` diff --git a/docs/generator_readiness_gap_registry_2026-02-26.md b/docs/generator_readiness_gap_registry_2026-02-26.md index ce2f2c1..47c98e7 100644 --- a/docs/generator_readiness_gap_registry_2026-02-26.md +++ b/docs/generator_readiness_gap_registry_2026-02-26.md @@ -47,6 +47,7 @@ This is the canonical dated registry for "not production-ready" generator gaps. | GR-018 | Performance/security/rollout constrained refactor enforcement gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/results.jsonl` | `partial` | Hard checks now enforce migration rollback + data-loss policy, security deny-by-default, SLO p95 presence, and rollout staged+abort policy. Enforcement is contract-level; generator capability under these constraints is still weak in C++ AB path. | Constraint-aware generation follow-up | | GR-019 | Parity-blocked readiness load (gating without capability closure) | `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r7/summary.json`, `logs/taskitem_runs/challenging_subset_prod_20260226_r7/summary.json`, `docs/sprint225_227_execution_tracker_2026-02-26.md` | `partial` | Sprint 225-227 reduced blocked parity load from `6` to `0` on both tracked hard catalogs while keeping unresolved divergence at `0`. This closes immediate safety debt for current corpora, but robustness is still contingent on pattern-driven repair classes. | Generalize repairs beyond queue-shaped transpile outputs | | GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`, `logs/taskitem_runs/TEST_ONLY_sprint239_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145631/00_summary.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md`, `docs/sprint238_execution_tracker_2026-02-26.md`, `docs/sprint239_execution_tracker_2026-02-26.md`, `docs/sprint240_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Sprint 238 added native decomposition hard gate in pipeline (`min task count`, `min semantic signal count`) so weak native generation can be blocked before fallback masking. Sprint 239 added native reason enrichment and improved semantic signal density (`0 -> 6`). Sprint 240 added native decomposition retry with explicit minimum-task policy; on current sample retry attempted but did not improve depth (`2 -> 2`). | Native decomposition task-depth upgrade (increase native task granularity beyond 2) | +| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json` | `partial` | Sprint 241 added one-by-one impact profiles and a pipeline-integrated native impact coverage gate (`native_impact_coverage`). Current fullstack sample activates 7 profiles and all fail, proving enforcement catches shallow coverage but generator closure is still pending. | Implement profile-specific native decomposition expansion to satisfy failing profile constraints | ## What Was Covered Today (Sprints 175-184) diff --git a/docs/native_decomposition_impact_list_2026-02-26.md b/docs/native_decomposition_impact_list_2026-02-26.md new file mode 100644 index 0000000..ee868fd --- /dev/null +++ b/docs/native_decomposition_impact_list_2026-02-26.md @@ -0,0 +1,68 @@ +# Native Decomposition Impact List - 2026-02-26 + +Status: Active +Purpose: Enumerate all known areas affected by shallow native decomposition and map each to enforceable tooling. + +## Impact List (One-by-One) + +1. API contract evolution +- Problem: compatibility/versioning work collapses into generic tasks. +- Tooling: `impact_api_contract_evolution` profile in `tools/mcp/profiles/native_decomposition_impact_profiles.json`. + +2. State migration and rollback safety +- Problem: migration/backfill/rollback choreography omitted. +- Tooling: `impact_state_migration_rollback` profile. + +3. Security policy propagation +- Problem: cross-layer auth/deny-by-default checks missing. +- Tooling: `impact_security_policy_propagation` profile. + +4. SLO/latency budget constraints +- Problem: performance requirements missing from decomposition rationale. +- Tooling: `impact_slo_latency_budget` profile. + +5. Rollout choreography +- Problem: staged rollout/abort hooks not represented as tasks. +- Tooling: `impact_rollout_choreography` profile. + +6. Multi-file transactional edits +- Problem: partial cross-file updates pass as “done”. +- Tooling: `impact_multifile_transactional_edit` profile. + +7. Concurrency/locking semantics +- Problem: race/lock/deadlock work under-scoped. +- Tooling: `impact_concurrency_locking` profile. + +8. Projection/environment constraints +- Problem: embedded/mobile/serverless constraints ignored. +- Tooling: `impact_projection_environment_constraints` profile. + +9. Polyglot boundary changes +- Problem: SDK/client/server coupling not decomposed. +- Tooling: `impact_polyglot_boundary_changes` profile. + +10. Observability/telemetry integrity +- Problem: logging/metrics/tracing semantics omitted. +- Tooling: `impact_observability_telemetry_integrity` profile. + +11. Data integrity and precision +- Problem: financial/scientific precision risks hidden in generic tasks. +- Tooling: `impact_data_integrity_precision` profile. + +12. Recovery/replay-critical flows +- Problem: recovery/rollback/replay pathways not represented. +- Tooling: `impact_disaster_recovery_replay` profile. + +## Enforcement Tooling + +- Checker: + - `tools/mcp/check_native_decomposition_impact_coverage.py` +- Profiles: + - `tools/mcp/profiles/native_decomposition_impact_profiles.json` +- Standalone gate wrapper: + - `tools/mcp/run_native_impact_coverage_gate.sh` +- Pipeline integration toggles in `tools/mcp/run_sprint_taskitem_pipeline.sh`: + - `WSTONE_NATIVE_IMPACT_COVERAGE_GATE` + - `WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE` + - `WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES` + diff --git a/docs/progress_log_2026-02-26.md b/docs/progress_log_2026-02-26.md index f143191..3ec8ea3 100644 --- a/docs/progress_log_2026-02-26.md +++ b/docs/progress_log_2026-02-26.md @@ -185,3 +185,22 @@ - `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145632/00_summary.json` - Current measured result: - retry attempted but not applied (`2 -> 2` tasks), confirming remaining native decomposition depth gap. + +## Sprint 241 Added (Same Day) + +- Added complete dated impact list for shallow native decomposition: + - `docs/native_decomposition_impact_list_2026-02-26.md` +- Added one-by-one profile tooling: + - `tools/mcp/profiles/native_decomposition_impact_profiles.json` + - `tools/mcp/check_native_decomposition_impact_coverage.py` + - `tools/mcp/run_native_impact_coverage_gate.sh` + - `tools/mcp/analyze_native_impact_coverage.py` +- Integrated impact-coverage gate into pipeline with optional enforcement: + - `WSTONE_NATIVE_IMPACT_COVERAGE_GATE` + - `WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE` + - `WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES` +- Dated artifacts: + - `logs/taskitem_runs/TEST_ONLY_sprint241_impact_gate_20260226_150403/00_summary.json` + - `logs/taskitem_runs/TEST_ONLY_sprint241_impact_gate_20260226_150404/06_native_impact_coverage.json` + - `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json` + - `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json` diff --git a/docs/sprint241_execution_tracker_2026-02-26.md b/docs/sprint241_execution_tracker_2026-02-26.md new file mode 100644 index 0000000..bd05683 --- /dev/null +++ b/docs/sprint241_execution_tracker_2026-02-26.md @@ -0,0 +1,55 @@ +# Sprint 241 Execution Tracker - 2026-02-26 + +## Scope +- `sprint241_plan.md` + +## Implemented + +Impact list (dated): +- `docs/native_decomposition_impact_list_2026-02-26.md` + +Profile set (one-by-one): +- `tools/mcp/profiles/native_decomposition_impact_profiles.json` +- profiles cover 12 impact surfaces: + - API contract evolution + - state migration/rollback + - security propagation + - SLO/latency budget + - rollout choreography + - multi-file transactional edit + - concurrency/locking + - projection/environment constraints + - polyglot boundary changes + - observability/telemetry integrity + - data integrity/precision + - recovery/replay critical flows + +Checker + wrappers: +- `tools/mcp/check_native_decomposition_impact_coverage.py` +- `tools/mcp/run_native_impact_coverage_gate.sh` +- `tools/mcp/analyze_native_impact_coverage.py` + +Pipeline integration in `tools/mcp/run_sprint_taskitem_pipeline.sh`: +- `WSTONE_NATIVE_IMPACT_COVERAGE_GATE` (default `0`) +- `WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE` (default `0`) +- `WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES` (profile path) +- emits `06_native_impact_coverage.json` +- summary includes `native_impact_coverage` + +## Baseline Validation Artifacts + +- soft-gate run: + - `logs/taskitem_runs/TEST_ONLY_sprint241_impact_gate_20260226_150403/00_summary.json` +- hard-gate run (expected block): + - `logs/taskitem_runs/TEST_ONLY_sprint241_impact_gate_20260226_150404/06_native_impact_coverage.json` +- multi-profile fullstack sample: + - `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json` +- aggregate report: + - `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json` + +Observed signal: +- Fullstack sample activated `7` profiles, all failing under current shallow native decomposition. + +## Explicit Completion Signal + +- Sprint 241: `DONE` (implemented + pipeline-integrated + enforced smoke verified) diff --git a/editor/src/Sprint241IntegrationSummary.h b/editor/src/Sprint241IntegrationSummary.h new file mode 100644 index 0000000..9773f00 --- /dev/null +++ b/editor/src/Sprint241IntegrationSummary.h @@ -0,0 +1,6 @@ +#pragma once + +// Sprint 241 integration summary: +// - Added comprehensive shallow-impact list and profile set. +// - Added profile-driven native decomposition coverage checker and gate wrapper. +// - Integrated optional impact coverage gating into sprint pipeline. diff --git a/sprint241_plan.md b/sprint241_plan.md new file mode 100644 index 0000000..136b360 --- /dev/null +++ b/sprint241_plan.md @@ -0,0 +1,12 @@ +# Sprint 241 Plan: Impact-by-Impact Native Decomposition Coverage Gates + +## Goal +Enumerate all major shallow-decomposition impact surfaces and enforce profile-specific native coverage checks for each. + +## Steps +- Step 2248: Create complete dated impact list for shallow decomposition effects. +- Step 2249: Define one-by-one impact profiles with trigger + required coverage constraints. +- Step 2250: Implement native impact coverage checker over run artifacts. +- Step 2251: Add standalone gate wrapper and aggregate analyzer. +- Step 2252: Integrate optional impact coverage gate into sprint pipeline. +- Step 2253: Add `Sprint241IntegrationSummary.h` and execution tracker. diff --git a/tools/mcp/analyze_native_impact_coverage.py b/tools/mcp/analyze_native_impact_coverage.py new file mode 100755 index 0000000..555cfe2 --- /dev/null +++ b/tools/mcp/analyze_native_impact_coverage.py @@ -0,0 +1,61 @@ +#!/usr/bin/env python3 +import argparse +import json +from pathlib import Path +from typing import Dict + + +def load_json(path: Path) -> Dict: + with path.open("r", encoding="utf-8") as f: + return json.load(f) + + +def write_json(path: Path, obj: Dict) -> None: + with path.open("w", encoding="utf-8") as f: + json.dump(obj, f, indent=2, sort_keys=True) + f.write("\n") + + +def main() -> int: + p = argparse.ArgumentParser(description="Aggregate native impact coverage reports across runs.") + p.add_argument("--runs-root", default="logs/taskitem_runs") + p.add_argument("--include-glob", default="*") + p.add_argument("--out", required=True) + args = p.parse_args() + + root = Path(args.runs_root) + profile_counts: Dict[str, Dict[str, int]] = {} + total_runs = 0 + failed_runs = 0 + + for report_path in sorted(root.glob(f"{args.include_glob}/06_native_impact_coverage.json")): + try: + report = load_json(report_path) + except Exception: + continue + total_runs += 1 + if report.get("status") == "fail": + failed_runs += 1 + for c in report.get("checks", []): + pid = str(c.get("id", "unknown")) + row = profile_counts.setdefault(pid, {"active": 0, "passed": 0, "failed": 0}) + row["active"] += 1 + if c.get("passed", False): + row["passed"] += 1 + else: + row["failed"] += 1 + + out = { + "status": "ok", + "total_runs": total_runs, + "failed_runs": failed_runs, + "fail_rate": round((failed_runs / total_runs), 4) if total_runs else 0.0, + "profile_counts": profile_counts, + } + write_json(Path(args.out), out) + print(json.dumps({"status": "ok", "total_runs": total_runs, "failed_runs": failed_runs, "out": args.out}, sort_keys=True)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/mcp/check_native_decomposition_impact_coverage.py b/tools/mcp/check_native_decomposition_impact_coverage.py new file mode 100755 index 0000000..05e45e6 --- /dev/null +++ b/tools/mcp/check_native_decomposition_impact_coverage.py @@ -0,0 +1,166 @@ +#!/usr/bin/env python3 +import argparse +import json +from pathlib import Path +from typing import Dict, List, Tuple + + +def load_json(path: Path) -> Dict: + with path.open("r", encoding="utf-8") as f: + return json.load(f) + + +def write_json(path: Path, obj: Dict) -> None: + with path.open("w", encoding="utf-8") as f: + json.dump(obj, f, indent=2, sort_keys=True) + f.write("\n") + + +def load_native_tasks(run_dir: Path, summary: Dict) -> List[Dict]: + retry = summary.get("native_decomposition_retry") or {} + retry_applied = bool(retry.get("applied", False)) + retry_path = run_dir / "02aa_generate_taskitems_retry.json" + base_path = run_dir / "02_generate_taskitems.json" + if retry_applied and retry_path.exists(): + data = load_json(retry_path) + if isinstance(data, dict): + return list(data.get("tasks") or []) + if base_path.exists(): + data = load_json(base_path) + if isinstance(data, dict): + return list(data.get("tasks") or []) + return [] + + +def load_spec_text(summary: Dict, repo_root: Path) -> str: + input_file = str(summary.get("input_file", "")).strip() + if not input_file: + return "" + p = Path(input_file) + if not p.is_absolute(): + p = repo_root / input_file + try: + return p.read_text(encoding="utf-8", errors="ignore") + except Exception: + return "" + + +def activate_profiles(profiles: List[Dict], spec_text: str) -> List[Dict]: + s = spec_text.lower() + active = [] + for profile in profiles: + keys = [str(k).lower() for k in (profile.get("trigger_keywords") or [])] + if not keys: + continue + if any(k in s for k in keys): + active.append(profile) + return active + + +def collect_task_unions(tasks: List[Dict]) -> Tuple[List[str], List[str], List[Dict]]: + reasons: List[str] = [] + prereq_ops: List[str] = [] + contracts: List[Dict] = [] + for t in tasks: + rs = t.get("reasons") or [] + po = t.get("prerequisiteOps") or [] + ec = t.get("executionContract") or {} + for r in rs: + reasons.append(str(r).lower()) + for op in po: + prereq_ops.append(str(op)) + if isinstance(ec, dict): + contracts.append(ec) + return reasons, prereq_ops, contracts + + +def check_profile(profile: Dict, tasks: List[Dict]) -> Dict: + reasons, prereq_ops, contracts = collect_task_unions(tasks) + missing: List[str] = [] + + min_tasks = int(profile.get("min_native_task_count", 0) or 0) + if len(tasks) < min_tasks: + missing.append(f"native_task_count<{min_tasks}") + + req_reason_keys = [str(x).lower() for x in (profile.get("required_reason_keywords") or [])] + for key in req_reason_keys: + if not any(key in r for r in reasons): + missing.append(f"missing_reason_keyword:{key}") + + req_ops = [str(x) for x in (profile.get("required_prerequisite_ops") or [])] + ops_set = set(prereq_ops) + for op in req_ops: + if op not in ops_set: + missing.append(f"missing_prerequisite_op:{op}") + + req_contract = [str(x) for x in (profile.get("required_execution_contract") or [])] + for field in req_contract: + has_field_true = False + for ec in contracts: + if isinstance(ec.get(field), bool): + if ec.get(field) is True: + has_field_true = True + break + elif field in ec: + has_field_true = True + break + if not has_field_true: + missing.append(f"missing_execution_contract:{field}") + + return { + "id": profile.get("id", "unknown"), + "description": profile.get("description", ""), + "passed": len(missing) == 0, + "native_task_count": len(tasks), + "missing": missing, + } + + +def main() -> int: + parser = argparse.ArgumentParser(description="Check native decomposition coverage against impact profiles.") + parser.add_argument("--run-dir", required=True, help="Pipeline run directory containing 00_summary.json") + parser.add_argument("--profiles", default="tools/mcp/profiles/native_decomposition_impact_profiles.json", help="Profile JSON path") + parser.add_argument("--out", required=True, help="Output report path") + parser.add_argument("--enforce", action="store_true", help="Exit nonzero on failures") + args = parser.parse_args() + + run_dir = Path(args.run_dir) + summary_path = run_dir / "00_summary.json" + if not summary_path.exists(): + raise SystemExit(f"missing summary: {summary_path}") + + summary = load_json(summary_path) + profiles_doc = load_json(Path(args.profiles)) + profiles = list(profiles_doc.get("profiles") or []) + repo_root = Path(__file__).resolve().parents[2] + spec_text = load_spec_text(summary, repo_root) + + active = activate_profiles(profiles, spec_text) + tasks = load_native_tasks(run_dir, summary) + + checks = [check_profile(p, tasks) for p in active] + failing = [c for c in checks if not c.get("passed", False)] + + result = { + "status": "ok" if not failing else "fail", + "run_dir": str(run_dir), + "active_profile_count": len(active), + "failing_profile_count": len(failing), + "active_profiles": [p.get("id", "unknown") for p in active], + "checks": checks, + } + write_json(Path(args.out), result) + print(json.dumps({ + "status": result["status"], + "active_profile_count": result["active_profile_count"], + "failing_profile_count": result["failing_profile_count"], + "out": args.out, + }, sort_keys=True)) + + if args.enforce and failing: + return 12 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/mcp/profiles/native_decomposition_impact_profiles.json b/tools/mcp/profiles/native_decomposition_impact_profiles.json new file mode 100644 index 0000000..7fde82c --- /dev/null +++ b/tools/mcp/profiles/native_decomposition_impact_profiles.json @@ -0,0 +1,113 @@ +{ + "schema": "native_decomposition_impact_profiles_v1", + "profiles": [ + { + "id": "impact_api_contract_evolution", + "description": "API/schema evolution with compatibility obligations.", + "trigger_keywords": ["api", "openapi", "schema", "contract", "compatibility", "version"], + "min_native_task_count": 5, + "required_reason_keywords": ["contract", "compatibility"], + "required_prerequisite_ops": ["whetstone_generate_taskitems", "whetstone_validate_taskitem", "whetstone_queue_ready"], + "required_execution_contract": ["deterministic", "rollbackRequired", "replayValidationRequired"] + }, + { + "id": "impact_state_migration_rollback", + "description": "State/data migrations that require rollback and data safety choreography.", + "trigger_keywords": ["migration", "rollback", "backfill", "data loss", "compatibility window"], + "min_native_task_count": 6, + "required_reason_keywords": ["migration", "risk"], + "required_prerequisite_ops": ["whetstone_generate_taskitems", "whetstone_validate_taskitem", "whetstone_queue_ready"], + "required_execution_contract": ["deterministic", "rollbackRequired", "replayValidationRequired"] + }, + { + "id": "impact_security_policy_propagation", + "description": "Cross-layer security policy propagation and deny-by-default handling.", + "trigger_keywords": ["security", "auth", "permission", "deny-by-default", "threat"], + "min_native_task_count": 5, + "required_reason_keywords": ["security", "risk"], + "required_prerequisite_ops": ["whetstone_validate_taskitem", "whetstone_queue_ready"], + "required_execution_contract": ["deterministic", "rollbackRequired"] + }, + { + "id": "impact_slo_latency_budget", + "description": "Performance/SLO constrained changes.", + "trigger_keywords": ["latency", "slo", "p95", "throughput", "budget"], + "min_native_task_count": 4, + "required_reason_keywords": ["performance", "risk"], + "required_prerequisite_ops": ["whetstone_validate_taskitem"], + "required_execution_contract": ["deterministic", "replayValidationRequired"] + }, + { + "id": "impact_rollout_choreography", + "description": "Feature flags, staged rollout, and abort choreography.", + "trigger_keywords": ["rollout", "stage", "feature flag", "canary", "abort"], + "min_native_task_count": 5, + "required_reason_keywords": ["rollout", "risk"], + "required_prerequisite_ops": ["whetstone_queue_ready", "whetstone_validate_taskitem"], + "required_execution_contract": ["deterministic", "rollbackRequired"] + }, + { + "id": "impact_multifile_transactional_edit", + "description": "Cross-file transactional edits where partial updates are unsafe.", + "trigger_keywords": ["multi-file", "cross-file", "transaction", "refactor", "across files"], + "min_native_task_count": 6, + "required_reason_keywords": ["transaction", "contract"], + "required_prerequisite_ops": ["whetstone_queue_ready", "whetstone_validate_taskitem"], + "required_execution_contract": ["deterministic", "rollbackRequired", "replayValidationRequired"] + }, + { + "id": "impact_concurrency_locking", + "description": "Threading, locking, races, and scheduling semantics.", + "trigger_keywords": ["concurrency", "thread", "mutex", "lock", "race", "deadlock"], + "min_native_task_count": 5, + "required_reason_keywords": ["concurrency", "risk"], + "required_prerequisite_ops": ["whetstone_validate_taskitem", "whetstone_queue_ready"], + "required_execution_contract": ["deterministic", "replayValidationRequired"] + }, + { + "id": "impact_projection_environment_constraints", + "description": "Target projection/environment constraints (embedded/mobile/serverless/hardware limits).", + "trigger_keywords": ["arduino", "embedded", "mobile", "ios", "android", "serverless", "memory limit", "hardware"], + "min_native_task_count": 4, + "required_reason_keywords": ["capability", "constraint"], + "required_prerequisite_ops": ["whetstone_architect_intake", "whetstone_validate_taskitem"], + "required_execution_contract": ["deterministic"] + }, + { + "id": "impact_polyglot_boundary_changes", + "description": "Cross-language boundary and API client/server regeneration work.", + "trigger_keywords": ["polyglot", "cross-language", "sdk", "client", "backend", "frontend"], + "min_native_task_count": 5, + "required_reason_keywords": ["compatibility", "contract"], + "required_prerequisite_ops": ["whetstone_generate_taskitems", "whetstone_validate_taskitem"], + "required_execution_contract": ["deterministic", "replayValidationRequired"] + }, + { + "id": "impact_observability_telemetry_integrity", + "description": "Logging/metrics/tracing changes requiring consistent telemetry semantics.", + "trigger_keywords": ["observability", "telemetry", "metrics", "tracing", "logging"], + "min_native_task_count": 4, + "required_reason_keywords": ["semantic", "contract"], + "required_prerequisite_ops": ["whetstone_validate_taskitem"], + "required_execution_contract": ["deterministic"] + }, + { + "id": "impact_data_integrity_precision", + "description": "Financial/scientific precision and data-integrity sensitive changes.", + "trigger_keywords": ["financial", "precision", "scientific", "numerical", "rounding"], + "min_native_task_count": 5, + "required_reason_keywords": ["risk", "contract"], + "required_prerequisite_ops": ["whetstone_validate_taskitem"], + "required_execution_contract": ["deterministic", "replayValidationRequired"] + }, + { + "id": "impact_disaster_recovery_replay", + "description": "Recovery/replay/rollback critical workflows.", + "trigger_keywords": ["recovery", "replay", "rollback", "incident", "failover"], + "min_native_task_count": 5, + "required_reason_keywords": ["risk", "replay"], + "required_prerequisite_ops": ["whetstone_validate_taskitem", "whetstone_queue_ready"], + "required_execution_contract": ["deterministic", "rollbackRequired", "replayValidationRequired"] + } + ] +} diff --git a/tools/mcp/run_native_impact_coverage_gate.sh b/tools/mcp/run_native_impact_coverage_gate.sh new file mode 100755 index 0000000..86fd8ba --- /dev/null +++ b/tools/mcp/run_native_impact_coverage_gate.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +RUN_DIR="${1:-}" +if [[ -z "$RUN_DIR" ]]; then + echo "usage: $0 [out_json]" >&2 + exit 2 +fi +OUT_JSON="${2:-$RUN_DIR/06_native_impact_coverage.json}" +PROFILES="${WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES:-$ROOT_DIR/tools/mcp/profiles/native_decomposition_impact_profiles.json}" +ENFORCE="${WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE:-1}" + +args=( + --run-dir "$RUN_DIR" + --profiles "$PROFILES" + --out "$OUT_JSON" +) +if [[ "$ENFORCE" == "1" ]]; then + args+=(--enforce) +fi + +if python3 "$ROOT_DIR/tools/mcp/check_native_decomposition_impact_coverage.py" "${args[@]}" >/dev/null; then + jq -n --arg run_dir "$RUN_DIR" --argjson report "$(cat "$OUT_JSON")" '{status:"pass", run_dir:$run_dir, report:$report}' +else + jq -n --arg run_dir "$RUN_DIR" --argjson report "$(cat "$OUT_JSON")" '{status:"fail", run_dir:$run_dir, report:$report}' + exit 12 +fi diff --git a/tools/mcp/run_sprint_taskitem_pipeline.sh b/tools/mcp/run_sprint_taskitem_pipeline.sh index 173b574..482cfd4 100755 --- a/tools/mcp/run_sprint_taskitem_pipeline.sh +++ b/tools/mcp/run_sprint_taskitem_pipeline.sh @@ -39,6 +39,9 @@ NATIVE_SEMANTIC_SIGNAL_MIN="${WSTONE_NATIVE_SEMANTIC_SIGNAL_MIN:-0}" NATIVE_REASON_ENRICHMENT="${WSTONE_NATIVE_REASON_ENRICHMENT:-1}" NATIVE_DECOMP_RETRY="${WSTONE_NATIVE_DECOMP_RETRY:-1}" NATIVE_DECOMP_TARGET_MIN_TASKS="${WSTONE_NATIVE_DECOMP_TARGET_MIN_TASKS:-5}" +NATIVE_IMPACT_COVERAGE_GATE="${WSTONE_NATIVE_IMPACT_COVERAGE_GATE:-0}" +NATIVE_IMPACT_COVERAGE_ENFORCE="${WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE:-0}" +NATIVE_IMPACT_COVERAGE_PROFILES="${WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES:-$ROOT_DIR/tools/mcp/profiles/native_decomposition_impact_profiles.json}" CAPABILITY_SIGNALS_JSON="${WSTONE_CAPABILITY_SIGNALS_JSON:-}" if [[ -z "$CAPABILITY_SIGNALS_JSON" ]]; then CAPABILITY_SIGNALS_JSON='{}' @@ -93,6 +96,7 @@ SEMANTIC_GATE_JSON='{}' NATIVE_DECOMP_GATE_JSON='{}' NATIVE_REASON_ENRICHMENT_JSON='{}' NATIVE_DECOMP_RETRY_JSON='{}' +NATIVE_IMPACT_COVERAGE_JSON='{}' if [[ "$SEMANTIC_PLANNING_BRIDGE" == "1" ]]; then python3 "$ROOT_DIR/tools/mcp/markdown_to_semantic_annotations.py" \ --spec "$INPUT_FILE" \ @@ -571,6 +575,26 @@ SUMMARY_JSON="$(jq -nc \ } }')" +if [[ "$NATIVE_IMPACT_COVERAGE_GATE" == "1" ]]; then + # Emit a provisional summary so coverage checker can read run metadata. + printf '%s\n' "$SUMMARY_JSON" > "$OUT_DIR/00_summary.json" + if python3 "$ROOT_DIR/tools/mcp/check_native_decomposition_impact_coverage.py" \ + --run-dir "$OUT_DIR" \ + --profiles "$NATIVE_IMPACT_COVERAGE_PROFILES" \ + --out "$OUT_DIR/06_native_impact_coverage.json" \ + $([[ "$NATIVE_IMPACT_COVERAGE_ENFORCE" == "1" ]] && echo --enforce) >/dev/null; then + NATIVE_IMPACT_COVERAGE_JSON="$(cat "$OUT_DIR/06_native_impact_coverage.json")" + else + NATIVE_IMPACT_COVERAGE_JSON="$(cat "$OUT_DIR/06_native_impact_coverage.json")" + SUMMARY_JSON="$(printf '%s' "$SUMMARY_JSON" | jq --argjson nic "$NATIVE_IMPACT_COVERAGE_JSON" '.native_impact_coverage = $nic')" + printf '%s\n' "$SUMMARY_JSON" > "$OUT_DIR/00_summary.json" + echo "error: native impact coverage gate failed" >&2 + echo "error: see $OUT_DIR/06_native_impact_coverage.json" >&2 + exit 11 + fi + SUMMARY_JSON="$(printf '%s' "$SUMMARY_JSON" | jq --argjson nic "$NATIVE_IMPACT_COVERAGE_JSON" '.native_impact_coverage = $nic')" +fi + if [[ "$CALIBRATE_AFTER_RUN" == "1" ]]; then CALIBRATION_OUT_DIR="$OUT_DIR/calibration" CALIBRATION_JSON="$(python3 "$ROOT_DIR/tools/mcp/analyze_taskitem_calibration.py" \