Add native decomposition quality gate to pipeline (sprint 238)
This commit is contained in:
@@ -150,7 +150,7 @@ Parallel planning tranche (active):
|
||||
- `logs/taskitem_runs/TEST_ONLY_spec_planning_baseline_20260226/deterministic_spec_readiness.json`
|
||||
|
||||
Planning runtime controls (active):
|
||||
- `sprint232_plan.md` to `sprint237_plan.md`
|
||||
- `sprint232_plan.md` to `sprint238_plan.md`
|
||||
- semantic bridge + intake augmentation + requirement injection + expansion gating are wired into:
|
||||
- `tools/mcp/run_sprint_taskitem_pipeline.sh`
|
||||
- current policy default is native-first semantic fallback:
|
||||
@@ -160,3 +160,5 @@ Planning runtime controls (active):
|
||||
- fallback budget gating is available via:
|
||||
- `tools/mcp/check_semantic_fallback_budget.py`
|
||||
- `tools/mcp/run_semantic_fallback_budget_gate.sh`
|
||||
- native decomposition quality gate is available in pipeline summary/hard-fail path:
|
||||
- `native_decomposition_gate`
|
||||
|
||||
@@ -46,7 +46,7 @@ This is the canonical dated registry for "not production-ready" generator gaps.
|
||||
| GR-017 | Full-stack contract propagation gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/fullstack_contract_closure.json` | `partial` | Full-stack contract packet + strict validation added in benchmark orchestration and summary aggregation. Contract invalid rate is `0%` on valid catalog, but cross-layer semantic consistency is not yet equivalence-verified. | Semantic propagation verifier sprint |
|
||||
| GR-018 | Performance/security/rollout constrained refactor enforcement gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/results.jsonl` | `partial` | Hard checks now enforce migration rollback + data-loss policy, security deny-by-default, SLO p95 presence, and rollout staged+abort policy. Enforcement is contract-level; generator capability under these constraints is still weak in C++ AB path. | Constraint-aware generation follow-up |
|
||||
| GR-019 | Parity-blocked readiness load (gating without capability closure) | `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r7/summary.json`, `logs/taskitem_runs/challenging_subset_prod_20260226_r7/summary.json`, `docs/sprint225_227_execution_tracker_2026-02-26.md` | `partial` | Sprint 225-227 reduced blocked parity load from `6` to `0` on both tracked hard catalogs while keeping unresolved divergence at `0`. This closes immediate safety debt for current corpora, but robustness is still contingent on pattern-driven repair classes. | Generalize repairs beyond queue-shaped transpile outputs |
|
||||
| GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Current measured slice remains `fallback_rate=1.0`, so gating now detects but does not yet fix root capability gaps. | Native decomposition upgrade + semantic rationale enforcement |
|
||||
| GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md`, `docs/sprint238_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Sprint 238 added native decomposition hard gate in pipeline (`min task count`, `min semantic signal count`) so weak native generation can be blocked before fallback masking. Current measured slice still shows weak native decomposition (`2 tasks`, `0 semantic signals`) and needs generator-quality uplift to close. | Native decomposition upgrade + semantic rationale enforcement |
|
||||
|
||||
## What Was Covered Today (Sprints 175-184)
|
||||
|
||||
|
||||
@@ -147,3 +147,16 @@
|
||||
- Dated pass/fail artifacts captured:
|
||||
- fail (strict budget): `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`
|
||||
- pass (relaxed budget): `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_pass_20260226/semantic_fallback_budget_gate.json`
|
||||
|
||||
## Sprint 238 Added (Same Day)
|
||||
|
||||
- Added native decomposition quality gate directly in pipeline:
|
||||
- `WSTONE_NATIVE_DECOMP_HARD_GATE`
|
||||
- `WSTONE_NATIVE_TASK_MIN_COUNT`
|
||||
- `WSTONE_NATIVE_SEMANTIC_SIGNAL_MIN`
|
||||
- Added run artifact and summary packet:
|
||||
- `02b_native_decomposition_gate.json`
|
||||
- `native_decomposition_gate`
|
||||
- Dated gate verification artifacts:
|
||||
- fail packet: `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`
|
||||
- pass packet: `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_pass_20260226.json`
|
||||
|
||||
28
docs/sprint238_execution_tracker_2026-02-26.md
Normal file
28
docs/sprint238_execution_tracker_2026-02-26.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# Sprint 238 Execution Tracker - 2026-02-26
|
||||
|
||||
## Scope
|
||||
- `sprint238_plan.md`
|
||||
|
||||
## Implemented
|
||||
|
||||
Pipeline updates in `tools/mcp/run_sprint_taskitem_pipeline.sh`:
|
||||
- Added native decomposition gate controls:
|
||||
- `WSTONE_NATIVE_DECOMP_HARD_GATE` (default `0`)
|
||||
- `WSTONE_NATIVE_TASK_MIN_COUNT` (default `0`)
|
||||
- `WSTONE_NATIVE_SEMANTIC_SIGNAL_MIN` (default `0`)
|
||||
- Added gate artifact output:
|
||||
- `02b_native_decomposition_gate.json`
|
||||
- Added summary packet:
|
||||
- `native_decomposition_gate`
|
||||
- Added hard-fail behavior when enabled and thresholds are not met.
|
||||
|
||||
## Baseline Gate Artifacts
|
||||
|
||||
- expected fail gate packet:
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`
|
||||
- expected pass gate packet:
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_pass_20260226.json`
|
||||
|
||||
## Explicit Completion Signal
|
||||
|
||||
- Sprint 238: `DONE` (implemented + hard-fail/pass smoke verified)
|
||||
Reference in New Issue
Block a user