Add impact-by-impact native decomposition coverage gates (sprint 241)
This commit is contained in:
@@ -150,7 +150,7 @@ Parallel planning tranche (active):
|
||||
- `logs/taskitem_runs/TEST_ONLY_spec_planning_baseline_20260226/deterministic_spec_readiness.json`
|
||||
|
||||
Planning runtime controls (active):
|
||||
- `sprint232_plan.md` to `sprint240_plan.md`
|
||||
- `sprint232_plan.md` to `sprint241_plan.md`
|
||||
- semantic bridge + intake augmentation + requirement injection + expansion gating are wired into:
|
||||
- `tools/mcp/run_sprint_taskitem_pipeline.sh`
|
||||
- current policy default is native-first semantic fallback:
|
||||
@@ -169,3 +169,8 @@ Planning runtime controls (active):
|
||||
- `WSTONE_NATIVE_DECOMP_RETRY`
|
||||
- `WSTONE_NATIVE_DECOMP_TARGET_MIN_TASKS`
|
||||
- `native_decomposition_retry`
|
||||
- impact-specific native decomposition coverage can be enforced:
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_GATE`
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE`
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES`
|
||||
- `native_impact_coverage`
|
||||
|
||||
@@ -47,6 +47,7 @@ This is the canonical dated registry for "not production-ready" generator gaps.
|
||||
| GR-018 | Performance/security/rollout constrained refactor enforcement gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/results.jsonl` | `partial` | Hard checks now enforce migration rollback + data-loss policy, security deny-by-default, SLO p95 presence, and rollout staged+abort policy. Enforcement is contract-level; generator capability under these constraints is still weak in C++ AB path. | Constraint-aware generation follow-up |
|
||||
| GR-019 | Parity-blocked readiness load (gating without capability closure) | `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r7/summary.json`, `logs/taskitem_runs/challenging_subset_prod_20260226_r7/summary.json`, `docs/sprint225_227_execution_tracker_2026-02-26.md` | `partial` | Sprint 225-227 reduced blocked parity load from `6` to `0` on both tracked hard catalogs while keeping unresolved divergence at `0`. This closes immediate safety debt for current corpora, but robustness is still contingent on pattern-driven repair classes. | Generalize repairs beyond queue-shaped transpile outputs |
|
||||
| GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`, `logs/taskitem_runs/TEST_ONLY_sprint239_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145631/00_summary.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md`, `docs/sprint238_execution_tracker_2026-02-26.md`, `docs/sprint239_execution_tracker_2026-02-26.md`, `docs/sprint240_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Sprint 238 added native decomposition hard gate in pipeline (`min task count`, `min semantic signal count`) so weak native generation can be blocked before fallback masking. Sprint 239 added native reason enrichment and improved semantic signal density (`0 -> 6`). Sprint 240 added native decomposition retry with explicit minimum-task policy; on current sample retry attempted but did not improve depth (`2 -> 2`). | Native decomposition task-depth upgrade (increase native task granularity beyond 2) |
|
||||
| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json` | `partial` | Sprint 241 added one-by-one impact profiles and a pipeline-integrated native impact coverage gate (`native_impact_coverage`). Current fullstack sample activates 7 profiles and all fail, proving enforcement catches shallow coverage but generator closure is still pending. | Implement profile-specific native decomposition expansion to satisfy failing profile constraints |
|
||||
|
||||
## What Was Covered Today (Sprints 175-184)
|
||||
|
||||
|
||||
68
docs/native_decomposition_impact_list_2026-02-26.md
Normal file
68
docs/native_decomposition_impact_list_2026-02-26.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Native Decomposition Impact List - 2026-02-26
|
||||
|
||||
Status: Active
|
||||
Purpose: Enumerate all known areas affected by shallow native decomposition and map each to enforceable tooling.
|
||||
|
||||
## Impact List (One-by-One)
|
||||
|
||||
1. API contract evolution
|
||||
- Problem: compatibility/versioning work collapses into generic tasks.
|
||||
- Tooling: `impact_api_contract_evolution` profile in `tools/mcp/profiles/native_decomposition_impact_profiles.json`.
|
||||
|
||||
2. State migration and rollback safety
|
||||
- Problem: migration/backfill/rollback choreography omitted.
|
||||
- Tooling: `impact_state_migration_rollback` profile.
|
||||
|
||||
3. Security policy propagation
|
||||
- Problem: cross-layer auth/deny-by-default checks missing.
|
||||
- Tooling: `impact_security_policy_propagation` profile.
|
||||
|
||||
4. SLO/latency budget constraints
|
||||
- Problem: performance requirements missing from decomposition rationale.
|
||||
- Tooling: `impact_slo_latency_budget` profile.
|
||||
|
||||
5. Rollout choreography
|
||||
- Problem: staged rollout/abort hooks not represented as tasks.
|
||||
- Tooling: `impact_rollout_choreography` profile.
|
||||
|
||||
6. Multi-file transactional edits
|
||||
- Problem: partial cross-file updates pass as “done”.
|
||||
- Tooling: `impact_multifile_transactional_edit` profile.
|
||||
|
||||
7. Concurrency/locking semantics
|
||||
- Problem: race/lock/deadlock work under-scoped.
|
||||
- Tooling: `impact_concurrency_locking` profile.
|
||||
|
||||
8. Projection/environment constraints
|
||||
- Problem: embedded/mobile/serverless constraints ignored.
|
||||
- Tooling: `impact_projection_environment_constraints` profile.
|
||||
|
||||
9. Polyglot boundary changes
|
||||
- Problem: SDK/client/server coupling not decomposed.
|
||||
- Tooling: `impact_polyglot_boundary_changes` profile.
|
||||
|
||||
10. Observability/telemetry integrity
|
||||
- Problem: logging/metrics/tracing semantics omitted.
|
||||
- Tooling: `impact_observability_telemetry_integrity` profile.
|
||||
|
||||
11. Data integrity and precision
|
||||
- Problem: financial/scientific precision risks hidden in generic tasks.
|
||||
- Tooling: `impact_data_integrity_precision` profile.
|
||||
|
||||
12. Recovery/replay-critical flows
|
||||
- Problem: recovery/rollback/replay pathways not represented.
|
||||
- Tooling: `impact_disaster_recovery_replay` profile.
|
||||
|
||||
## Enforcement Tooling
|
||||
|
||||
- Checker:
|
||||
- `tools/mcp/check_native_decomposition_impact_coverage.py`
|
||||
- Profiles:
|
||||
- `tools/mcp/profiles/native_decomposition_impact_profiles.json`
|
||||
- Standalone gate wrapper:
|
||||
- `tools/mcp/run_native_impact_coverage_gate.sh`
|
||||
- Pipeline integration toggles in `tools/mcp/run_sprint_taskitem_pipeline.sh`:
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_GATE`
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE`
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES`
|
||||
|
||||
@@ -185,3 +185,22 @@
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145632/00_summary.json`
|
||||
- Current measured result:
|
||||
- retry attempted but not applied (`2 -> 2` tasks), confirming remaining native decomposition depth gap.
|
||||
|
||||
## Sprint 241 Added (Same Day)
|
||||
|
||||
- Added complete dated impact list for shallow native decomposition:
|
||||
- `docs/native_decomposition_impact_list_2026-02-26.md`
|
||||
- Added one-by-one profile tooling:
|
||||
- `tools/mcp/profiles/native_decomposition_impact_profiles.json`
|
||||
- `tools/mcp/check_native_decomposition_impact_coverage.py`
|
||||
- `tools/mcp/run_native_impact_coverage_gate.sh`
|
||||
- `tools/mcp/analyze_native_impact_coverage.py`
|
||||
- Integrated impact-coverage gate into pipeline with optional enforcement:
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_GATE`
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE`
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES`
|
||||
- Dated artifacts:
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint241_impact_gate_20260226_150403/00_summary.json`
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint241_impact_gate_20260226_150404/06_native_impact_coverage.json`
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`
|
||||
|
||||
55
docs/sprint241_execution_tracker_2026-02-26.md
Normal file
55
docs/sprint241_execution_tracker_2026-02-26.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# Sprint 241 Execution Tracker - 2026-02-26
|
||||
|
||||
## Scope
|
||||
- `sprint241_plan.md`
|
||||
|
||||
## Implemented
|
||||
|
||||
Impact list (dated):
|
||||
- `docs/native_decomposition_impact_list_2026-02-26.md`
|
||||
|
||||
Profile set (one-by-one):
|
||||
- `tools/mcp/profiles/native_decomposition_impact_profiles.json`
|
||||
- profiles cover 12 impact surfaces:
|
||||
- API contract evolution
|
||||
- state migration/rollback
|
||||
- security propagation
|
||||
- SLO/latency budget
|
||||
- rollout choreography
|
||||
- multi-file transactional edit
|
||||
- concurrency/locking
|
||||
- projection/environment constraints
|
||||
- polyglot boundary changes
|
||||
- observability/telemetry integrity
|
||||
- data integrity/precision
|
||||
- recovery/replay critical flows
|
||||
|
||||
Checker + wrappers:
|
||||
- `tools/mcp/check_native_decomposition_impact_coverage.py`
|
||||
- `tools/mcp/run_native_impact_coverage_gate.sh`
|
||||
- `tools/mcp/analyze_native_impact_coverage.py`
|
||||
|
||||
Pipeline integration in `tools/mcp/run_sprint_taskitem_pipeline.sh`:
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_GATE` (default `0`)
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE` (default `0`)
|
||||
- `WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES` (profile path)
|
||||
- emits `06_native_impact_coverage.json`
|
||||
- summary includes `native_impact_coverage`
|
||||
|
||||
## Baseline Validation Artifacts
|
||||
|
||||
- soft-gate run:
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint241_impact_gate_20260226_150403/00_summary.json`
|
||||
- hard-gate run (expected block):
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint241_impact_gate_20260226_150404/06_native_impact_coverage.json`
|
||||
- multi-profile fullstack sample:
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`
|
||||
- aggregate report:
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`
|
||||
|
||||
Observed signal:
|
||||
- Fullstack sample activated `7` profiles, all failing under current shallow native decomposition.
|
||||
|
||||
## Explicit Completion Signal
|
||||
|
||||
- Sprint 241: `DONE` (implemented + pipeline-integrated + enforced smoke verified)
|
||||
6
editor/src/Sprint241IntegrationSummary.h
Normal file
6
editor/src/Sprint241IntegrationSummary.h
Normal file
@@ -0,0 +1,6 @@
|
||||
#pragma once
|
||||
|
||||
// Sprint 241 integration summary:
|
||||
// - Added comprehensive shallow-impact list and profile set.
|
||||
// - Added profile-driven native decomposition coverage checker and gate wrapper.
|
||||
// - Integrated optional impact coverage gating into sprint pipeline.
|
||||
12
sprint241_plan.md
Normal file
12
sprint241_plan.md
Normal file
@@ -0,0 +1,12 @@
|
||||
# Sprint 241 Plan: Impact-by-Impact Native Decomposition Coverage Gates
|
||||
|
||||
## Goal
|
||||
Enumerate all major shallow-decomposition impact surfaces and enforce profile-specific native coverage checks for each.
|
||||
|
||||
## Steps
|
||||
- Step 2248: Create complete dated impact list for shallow decomposition effects.
|
||||
- Step 2249: Define one-by-one impact profiles with trigger + required coverage constraints.
|
||||
- Step 2250: Implement native impact coverage checker over run artifacts.
|
||||
- Step 2251: Add standalone gate wrapper and aggregate analyzer.
|
||||
- Step 2252: Integrate optional impact coverage gate into sprint pipeline.
|
||||
- Step 2253: Add `Sprint241IntegrationSummary.h` and execution tracker.
|
||||
61
tools/mcp/analyze_native_impact_coverage.py
Executable file
61
tools/mcp/analyze_native_impact_coverage.py
Executable file
@@ -0,0 +1,61 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict
|
||||
|
||||
|
||||
def load_json(path: Path) -> Dict:
|
||||
with path.open("r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def write_json(path: Path, obj: Dict) -> None:
|
||||
with path.open("w", encoding="utf-8") as f:
|
||||
json.dump(obj, f, indent=2, sort_keys=True)
|
||||
f.write("\n")
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Aggregate native impact coverage reports across runs.")
|
||||
p.add_argument("--runs-root", default="logs/taskitem_runs")
|
||||
p.add_argument("--include-glob", default="*")
|
||||
p.add_argument("--out", required=True)
|
||||
args = p.parse_args()
|
||||
|
||||
root = Path(args.runs_root)
|
||||
profile_counts: Dict[str, Dict[str, int]] = {}
|
||||
total_runs = 0
|
||||
failed_runs = 0
|
||||
|
||||
for report_path in sorted(root.glob(f"{args.include_glob}/06_native_impact_coverage.json")):
|
||||
try:
|
||||
report = load_json(report_path)
|
||||
except Exception:
|
||||
continue
|
||||
total_runs += 1
|
||||
if report.get("status") == "fail":
|
||||
failed_runs += 1
|
||||
for c in report.get("checks", []):
|
||||
pid = str(c.get("id", "unknown"))
|
||||
row = profile_counts.setdefault(pid, {"active": 0, "passed": 0, "failed": 0})
|
||||
row["active"] += 1
|
||||
if c.get("passed", False):
|
||||
row["passed"] += 1
|
||||
else:
|
||||
row["failed"] += 1
|
||||
|
||||
out = {
|
||||
"status": "ok",
|
||||
"total_runs": total_runs,
|
||||
"failed_runs": failed_runs,
|
||||
"fail_rate": round((failed_runs / total_runs), 4) if total_runs else 0.0,
|
||||
"profile_counts": profile_counts,
|
||||
}
|
||||
write_json(Path(args.out), out)
|
||||
print(json.dumps({"status": "ok", "total_runs": total_runs, "failed_runs": failed_runs, "out": args.out}, sort_keys=True))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
166
tools/mcp/check_native_decomposition_impact_coverage.py
Executable file
166
tools/mcp/check_native_decomposition_impact_coverage.py
Executable file
@@ -0,0 +1,166 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Tuple
|
||||
|
||||
|
||||
def load_json(path: Path) -> Dict:
|
||||
with path.open("r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def write_json(path: Path, obj: Dict) -> None:
|
||||
with path.open("w", encoding="utf-8") as f:
|
||||
json.dump(obj, f, indent=2, sort_keys=True)
|
||||
f.write("\n")
|
||||
|
||||
|
||||
def load_native_tasks(run_dir: Path, summary: Dict) -> List[Dict]:
|
||||
retry = summary.get("native_decomposition_retry") or {}
|
||||
retry_applied = bool(retry.get("applied", False))
|
||||
retry_path = run_dir / "02aa_generate_taskitems_retry.json"
|
||||
base_path = run_dir / "02_generate_taskitems.json"
|
||||
if retry_applied and retry_path.exists():
|
||||
data = load_json(retry_path)
|
||||
if isinstance(data, dict):
|
||||
return list(data.get("tasks") or [])
|
||||
if base_path.exists():
|
||||
data = load_json(base_path)
|
||||
if isinstance(data, dict):
|
||||
return list(data.get("tasks") or [])
|
||||
return []
|
||||
|
||||
|
||||
def load_spec_text(summary: Dict, repo_root: Path) -> str:
|
||||
input_file = str(summary.get("input_file", "")).strip()
|
||||
if not input_file:
|
||||
return ""
|
||||
p = Path(input_file)
|
||||
if not p.is_absolute():
|
||||
p = repo_root / input_file
|
||||
try:
|
||||
return p.read_text(encoding="utf-8", errors="ignore")
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
def activate_profiles(profiles: List[Dict], spec_text: str) -> List[Dict]:
|
||||
s = spec_text.lower()
|
||||
active = []
|
||||
for profile in profiles:
|
||||
keys = [str(k).lower() for k in (profile.get("trigger_keywords") or [])]
|
||||
if not keys:
|
||||
continue
|
||||
if any(k in s for k in keys):
|
||||
active.append(profile)
|
||||
return active
|
||||
|
||||
|
||||
def collect_task_unions(tasks: List[Dict]) -> Tuple[List[str], List[str], List[Dict]]:
|
||||
reasons: List[str] = []
|
||||
prereq_ops: List[str] = []
|
||||
contracts: List[Dict] = []
|
||||
for t in tasks:
|
||||
rs = t.get("reasons") or []
|
||||
po = t.get("prerequisiteOps") or []
|
||||
ec = t.get("executionContract") or {}
|
||||
for r in rs:
|
||||
reasons.append(str(r).lower())
|
||||
for op in po:
|
||||
prereq_ops.append(str(op))
|
||||
if isinstance(ec, dict):
|
||||
contracts.append(ec)
|
||||
return reasons, prereq_ops, contracts
|
||||
|
||||
|
||||
def check_profile(profile: Dict, tasks: List[Dict]) -> Dict:
|
||||
reasons, prereq_ops, contracts = collect_task_unions(tasks)
|
||||
missing: List[str] = []
|
||||
|
||||
min_tasks = int(profile.get("min_native_task_count", 0) or 0)
|
||||
if len(tasks) < min_tasks:
|
||||
missing.append(f"native_task_count<{min_tasks}")
|
||||
|
||||
req_reason_keys = [str(x).lower() for x in (profile.get("required_reason_keywords") or [])]
|
||||
for key in req_reason_keys:
|
||||
if not any(key in r for r in reasons):
|
||||
missing.append(f"missing_reason_keyword:{key}")
|
||||
|
||||
req_ops = [str(x) for x in (profile.get("required_prerequisite_ops") or [])]
|
||||
ops_set = set(prereq_ops)
|
||||
for op in req_ops:
|
||||
if op not in ops_set:
|
||||
missing.append(f"missing_prerequisite_op:{op}")
|
||||
|
||||
req_contract = [str(x) for x in (profile.get("required_execution_contract") or [])]
|
||||
for field in req_contract:
|
||||
has_field_true = False
|
||||
for ec in contracts:
|
||||
if isinstance(ec.get(field), bool):
|
||||
if ec.get(field) is True:
|
||||
has_field_true = True
|
||||
break
|
||||
elif field in ec:
|
||||
has_field_true = True
|
||||
break
|
||||
if not has_field_true:
|
||||
missing.append(f"missing_execution_contract:{field}")
|
||||
|
||||
return {
|
||||
"id": profile.get("id", "unknown"),
|
||||
"description": profile.get("description", ""),
|
||||
"passed": len(missing) == 0,
|
||||
"native_task_count": len(tasks),
|
||||
"missing": missing,
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Check native decomposition coverage against impact profiles.")
|
||||
parser.add_argument("--run-dir", required=True, help="Pipeline run directory containing 00_summary.json")
|
||||
parser.add_argument("--profiles", default="tools/mcp/profiles/native_decomposition_impact_profiles.json", help="Profile JSON path")
|
||||
parser.add_argument("--out", required=True, help="Output report path")
|
||||
parser.add_argument("--enforce", action="store_true", help="Exit nonzero on failures")
|
||||
args = parser.parse_args()
|
||||
|
||||
run_dir = Path(args.run_dir)
|
||||
summary_path = run_dir / "00_summary.json"
|
||||
if not summary_path.exists():
|
||||
raise SystemExit(f"missing summary: {summary_path}")
|
||||
|
||||
summary = load_json(summary_path)
|
||||
profiles_doc = load_json(Path(args.profiles))
|
||||
profiles = list(profiles_doc.get("profiles") or [])
|
||||
repo_root = Path(__file__).resolve().parents[2]
|
||||
spec_text = load_spec_text(summary, repo_root)
|
||||
|
||||
active = activate_profiles(profiles, spec_text)
|
||||
tasks = load_native_tasks(run_dir, summary)
|
||||
|
||||
checks = [check_profile(p, tasks) for p in active]
|
||||
failing = [c for c in checks if not c.get("passed", False)]
|
||||
|
||||
result = {
|
||||
"status": "ok" if not failing else "fail",
|
||||
"run_dir": str(run_dir),
|
||||
"active_profile_count": len(active),
|
||||
"failing_profile_count": len(failing),
|
||||
"active_profiles": [p.get("id", "unknown") for p in active],
|
||||
"checks": checks,
|
||||
}
|
||||
write_json(Path(args.out), result)
|
||||
print(json.dumps({
|
||||
"status": result["status"],
|
||||
"active_profile_count": result["active_profile_count"],
|
||||
"failing_profile_count": result["failing_profile_count"],
|
||||
"out": args.out,
|
||||
}, sort_keys=True))
|
||||
|
||||
if args.enforce and failing:
|
||||
return 12
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
113
tools/mcp/profiles/native_decomposition_impact_profiles.json
Normal file
113
tools/mcp/profiles/native_decomposition_impact_profiles.json
Normal file
@@ -0,0 +1,113 @@
|
||||
{
|
||||
"schema": "native_decomposition_impact_profiles_v1",
|
||||
"profiles": [
|
||||
{
|
||||
"id": "impact_api_contract_evolution",
|
||||
"description": "API/schema evolution with compatibility obligations.",
|
||||
"trigger_keywords": ["api", "openapi", "schema", "contract", "compatibility", "version"],
|
||||
"min_native_task_count": 5,
|
||||
"required_reason_keywords": ["contract", "compatibility"],
|
||||
"required_prerequisite_ops": ["whetstone_generate_taskitems", "whetstone_validate_taskitem", "whetstone_queue_ready"],
|
||||
"required_execution_contract": ["deterministic", "rollbackRequired", "replayValidationRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_state_migration_rollback",
|
||||
"description": "State/data migrations that require rollback and data safety choreography.",
|
||||
"trigger_keywords": ["migration", "rollback", "backfill", "data loss", "compatibility window"],
|
||||
"min_native_task_count": 6,
|
||||
"required_reason_keywords": ["migration", "risk"],
|
||||
"required_prerequisite_ops": ["whetstone_generate_taskitems", "whetstone_validate_taskitem", "whetstone_queue_ready"],
|
||||
"required_execution_contract": ["deterministic", "rollbackRequired", "replayValidationRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_security_policy_propagation",
|
||||
"description": "Cross-layer security policy propagation and deny-by-default handling.",
|
||||
"trigger_keywords": ["security", "auth", "permission", "deny-by-default", "threat"],
|
||||
"min_native_task_count": 5,
|
||||
"required_reason_keywords": ["security", "risk"],
|
||||
"required_prerequisite_ops": ["whetstone_validate_taskitem", "whetstone_queue_ready"],
|
||||
"required_execution_contract": ["deterministic", "rollbackRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_slo_latency_budget",
|
||||
"description": "Performance/SLO constrained changes.",
|
||||
"trigger_keywords": ["latency", "slo", "p95", "throughput", "budget"],
|
||||
"min_native_task_count": 4,
|
||||
"required_reason_keywords": ["performance", "risk"],
|
||||
"required_prerequisite_ops": ["whetstone_validate_taskitem"],
|
||||
"required_execution_contract": ["deterministic", "replayValidationRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_rollout_choreography",
|
||||
"description": "Feature flags, staged rollout, and abort choreography.",
|
||||
"trigger_keywords": ["rollout", "stage", "feature flag", "canary", "abort"],
|
||||
"min_native_task_count": 5,
|
||||
"required_reason_keywords": ["rollout", "risk"],
|
||||
"required_prerequisite_ops": ["whetstone_queue_ready", "whetstone_validate_taskitem"],
|
||||
"required_execution_contract": ["deterministic", "rollbackRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_multifile_transactional_edit",
|
||||
"description": "Cross-file transactional edits where partial updates are unsafe.",
|
||||
"trigger_keywords": ["multi-file", "cross-file", "transaction", "refactor", "across files"],
|
||||
"min_native_task_count": 6,
|
||||
"required_reason_keywords": ["transaction", "contract"],
|
||||
"required_prerequisite_ops": ["whetstone_queue_ready", "whetstone_validate_taskitem"],
|
||||
"required_execution_contract": ["deterministic", "rollbackRequired", "replayValidationRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_concurrency_locking",
|
||||
"description": "Threading, locking, races, and scheduling semantics.",
|
||||
"trigger_keywords": ["concurrency", "thread", "mutex", "lock", "race", "deadlock"],
|
||||
"min_native_task_count": 5,
|
||||
"required_reason_keywords": ["concurrency", "risk"],
|
||||
"required_prerequisite_ops": ["whetstone_validate_taskitem", "whetstone_queue_ready"],
|
||||
"required_execution_contract": ["deterministic", "replayValidationRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_projection_environment_constraints",
|
||||
"description": "Target projection/environment constraints (embedded/mobile/serverless/hardware limits).",
|
||||
"trigger_keywords": ["arduino", "embedded", "mobile", "ios", "android", "serverless", "memory limit", "hardware"],
|
||||
"min_native_task_count": 4,
|
||||
"required_reason_keywords": ["capability", "constraint"],
|
||||
"required_prerequisite_ops": ["whetstone_architect_intake", "whetstone_validate_taskitem"],
|
||||
"required_execution_contract": ["deterministic"]
|
||||
},
|
||||
{
|
||||
"id": "impact_polyglot_boundary_changes",
|
||||
"description": "Cross-language boundary and API client/server regeneration work.",
|
||||
"trigger_keywords": ["polyglot", "cross-language", "sdk", "client", "backend", "frontend"],
|
||||
"min_native_task_count": 5,
|
||||
"required_reason_keywords": ["compatibility", "contract"],
|
||||
"required_prerequisite_ops": ["whetstone_generate_taskitems", "whetstone_validate_taskitem"],
|
||||
"required_execution_contract": ["deterministic", "replayValidationRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_observability_telemetry_integrity",
|
||||
"description": "Logging/metrics/tracing changes requiring consistent telemetry semantics.",
|
||||
"trigger_keywords": ["observability", "telemetry", "metrics", "tracing", "logging"],
|
||||
"min_native_task_count": 4,
|
||||
"required_reason_keywords": ["semantic", "contract"],
|
||||
"required_prerequisite_ops": ["whetstone_validate_taskitem"],
|
||||
"required_execution_contract": ["deterministic"]
|
||||
},
|
||||
{
|
||||
"id": "impact_data_integrity_precision",
|
||||
"description": "Financial/scientific precision and data-integrity sensitive changes.",
|
||||
"trigger_keywords": ["financial", "precision", "scientific", "numerical", "rounding"],
|
||||
"min_native_task_count": 5,
|
||||
"required_reason_keywords": ["risk", "contract"],
|
||||
"required_prerequisite_ops": ["whetstone_validate_taskitem"],
|
||||
"required_execution_contract": ["deterministic", "replayValidationRequired"]
|
||||
},
|
||||
{
|
||||
"id": "impact_disaster_recovery_replay",
|
||||
"description": "Recovery/replay/rollback critical workflows.",
|
||||
"trigger_keywords": ["recovery", "replay", "rollback", "incident", "failover"],
|
||||
"min_native_task_count": 5,
|
||||
"required_reason_keywords": ["risk", "replay"],
|
||||
"required_prerequisite_ops": ["whetstone_validate_taskitem", "whetstone_queue_ready"],
|
||||
"required_execution_contract": ["deterministic", "rollbackRequired", "replayValidationRequired"]
|
||||
}
|
||||
]
|
||||
}
|
||||
28
tools/mcp/run_native_impact_coverage_gate.sh
Executable file
28
tools/mcp/run_native_impact_coverage_gate.sh
Executable file
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
RUN_DIR="${1:-}"
|
||||
if [[ -z "$RUN_DIR" ]]; then
|
||||
echo "usage: $0 <run_dir> [out_json]" >&2
|
||||
exit 2
|
||||
fi
|
||||
OUT_JSON="${2:-$RUN_DIR/06_native_impact_coverage.json}"
|
||||
PROFILES="${WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES:-$ROOT_DIR/tools/mcp/profiles/native_decomposition_impact_profiles.json}"
|
||||
ENFORCE="${WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE:-1}"
|
||||
|
||||
args=(
|
||||
--run-dir "$RUN_DIR"
|
||||
--profiles "$PROFILES"
|
||||
--out "$OUT_JSON"
|
||||
)
|
||||
if [[ "$ENFORCE" == "1" ]]; then
|
||||
args+=(--enforce)
|
||||
fi
|
||||
|
||||
if python3 "$ROOT_DIR/tools/mcp/check_native_decomposition_impact_coverage.py" "${args[@]}" >/dev/null; then
|
||||
jq -n --arg run_dir "$RUN_DIR" --argjson report "$(cat "$OUT_JSON")" '{status:"pass", run_dir:$run_dir, report:$report}'
|
||||
else
|
||||
jq -n --arg run_dir "$RUN_DIR" --argjson report "$(cat "$OUT_JSON")" '{status:"fail", run_dir:$run_dir, report:$report}'
|
||||
exit 12
|
||||
fi
|
||||
@@ -39,6 +39,9 @@ NATIVE_SEMANTIC_SIGNAL_MIN="${WSTONE_NATIVE_SEMANTIC_SIGNAL_MIN:-0}"
|
||||
NATIVE_REASON_ENRICHMENT="${WSTONE_NATIVE_REASON_ENRICHMENT:-1}"
|
||||
NATIVE_DECOMP_RETRY="${WSTONE_NATIVE_DECOMP_RETRY:-1}"
|
||||
NATIVE_DECOMP_TARGET_MIN_TASKS="${WSTONE_NATIVE_DECOMP_TARGET_MIN_TASKS:-5}"
|
||||
NATIVE_IMPACT_COVERAGE_GATE="${WSTONE_NATIVE_IMPACT_COVERAGE_GATE:-0}"
|
||||
NATIVE_IMPACT_COVERAGE_ENFORCE="${WSTONE_NATIVE_IMPACT_COVERAGE_ENFORCE:-0}"
|
||||
NATIVE_IMPACT_COVERAGE_PROFILES="${WSTONE_NATIVE_IMPACT_COVERAGE_PROFILES:-$ROOT_DIR/tools/mcp/profiles/native_decomposition_impact_profiles.json}"
|
||||
CAPABILITY_SIGNALS_JSON="${WSTONE_CAPABILITY_SIGNALS_JSON:-}"
|
||||
if [[ -z "$CAPABILITY_SIGNALS_JSON" ]]; then
|
||||
CAPABILITY_SIGNALS_JSON='{}'
|
||||
@@ -93,6 +96,7 @@ SEMANTIC_GATE_JSON='{}'
|
||||
NATIVE_DECOMP_GATE_JSON='{}'
|
||||
NATIVE_REASON_ENRICHMENT_JSON='{}'
|
||||
NATIVE_DECOMP_RETRY_JSON='{}'
|
||||
NATIVE_IMPACT_COVERAGE_JSON='{}'
|
||||
if [[ "$SEMANTIC_PLANNING_BRIDGE" == "1" ]]; then
|
||||
python3 "$ROOT_DIR/tools/mcp/markdown_to_semantic_annotations.py" \
|
||||
--spec "$INPUT_FILE" \
|
||||
@@ -571,6 +575,26 @@ SUMMARY_JSON="$(jq -nc \
|
||||
}
|
||||
}')"
|
||||
|
||||
if [[ "$NATIVE_IMPACT_COVERAGE_GATE" == "1" ]]; then
|
||||
# Emit a provisional summary so coverage checker can read run metadata.
|
||||
printf '%s\n' "$SUMMARY_JSON" > "$OUT_DIR/00_summary.json"
|
||||
if python3 "$ROOT_DIR/tools/mcp/check_native_decomposition_impact_coverage.py" \
|
||||
--run-dir "$OUT_DIR" \
|
||||
--profiles "$NATIVE_IMPACT_COVERAGE_PROFILES" \
|
||||
--out "$OUT_DIR/06_native_impact_coverage.json" \
|
||||
$([[ "$NATIVE_IMPACT_COVERAGE_ENFORCE" == "1" ]] && echo --enforce) >/dev/null; then
|
||||
NATIVE_IMPACT_COVERAGE_JSON="$(cat "$OUT_DIR/06_native_impact_coverage.json")"
|
||||
else
|
||||
NATIVE_IMPACT_COVERAGE_JSON="$(cat "$OUT_DIR/06_native_impact_coverage.json")"
|
||||
SUMMARY_JSON="$(printf '%s' "$SUMMARY_JSON" | jq --argjson nic "$NATIVE_IMPACT_COVERAGE_JSON" '.native_impact_coverage = $nic')"
|
||||
printf '%s\n' "$SUMMARY_JSON" > "$OUT_DIR/00_summary.json"
|
||||
echo "error: native impact coverage gate failed" >&2
|
||||
echo "error: see $OUT_DIR/06_native_impact_coverage.json" >&2
|
||||
exit 11
|
||||
fi
|
||||
SUMMARY_JSON="$(printf '%s' "$SUMMARY_JSON" | jq --argjson nic "$NATIVE_IMPACT_COVERAGE_JSON" '.native_impact_coverage = $nic')"
|
||||
fi
|
||||
|
||||
if [[ "$CALIBRATE_AFTER_RUN" == "1" ]]; then
|
||||
CALIBRATION_OUT_DIR="$OUT_DIR/calibration"
|
||||
CALIBRATION_JSON="$(python3 "$ROOT_DIR/tools/mcp/analyze_taskitem_calibration.py" \
|
||||
|
||||
Reference in New Issue
Block a user