Add profile task-bundle remediation loop closure (sprint 243)
This commit is contained in:
@@ -150,7 +150,7 @@ Parallel planning tranche (active):
|
||||
- `logs/taskitem_runs/TEST_ONLY_spec_planning_baseline_20260226/deterministic_spec_readiness.json`
|
||||
|
||||
Planning runtime controls (active):
|
||||
- `sprint232_plan.md` to `sprint242_plan.md`
|
||||
- `sprint232_plan.md` to `sprint243_plan.md`
|
||||
- semantic bridge + intake augmentation + requirement injection + expansion gating are wired into:
|
||||
- `tools/mcp/run_sprint_taskitem_pipeline.sh`
|
||||
- current policy default is native-first semantic fallback:
|
||||
@@ -178,3 +178,6 @@ Planning runtime controls (active):
|
||||
- `WSTONE_EXTRA_NORMALIZED_REQUIREMENTS_FILE`
|
||||
- `tools/mcp/synthesize_native_impact_remediation_requirements.py`
|
||||
- `tools/mcp/run_native_impact_remediation_loop.sh`
|
||||
- remediation loop can also synthesize and inject profile-specific task bundles:
|
||||
- `WSTONE_EXTRA_TASKS_FILE`
|
||||
- `tools/mcp/synthesize_native_impact_remediation_tasks.py`
|
||||
|
||||
@@ -47,7 +47,7 @@ This is the canonical dated registry for "not production-ready" generator gaps.
|
||||
| GR-018 | Performance/security/rollout constrained refactor enforcement gap | `docs/gap_hunt_fullstack_multifile_2026-02-26.md`, `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r5/results.jsonl` | `partial` | Hard checks now enforce migration rollback + data-loss policy, security deny-by-default, SLO p95 presence, and rollout staged+abort policy. Enforcement is contract-level; generator capability under these constraints is still weak in C++ AB path. | Constraint-aware generation follow-up |
|
||||
| GR-019 | Parity-blocked readiness load (gating without capability closure) | `logs/taskitem_runs/challenging_fullstack_multifile_20260226_r7/summary.json`, `logs/taskitem_runs/challenging_subset_prod_20260226_r7/summary.json`, `docs/sprint225_227_execution_tracker_2026-02-26.md` | `partial` | Sprint 225-227 reduced blocked parity load from `6` to `0` on both tracked hard catalogs while keeping unresolved divergence at `0`. This closes immediate safety debt for current corpora, but robustness is still contingent on pattern-driven repair classes. | Generalize repairs beyond queue-shaped transpile outputs |
|
||||
| GR-020 | Semantic fallback overuse masks weak native decomposition | `logs/taskitem_runs/TEST_ONLY_sprint236_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint237_semantic_fallback_gate_fail_20260226/semantic_fallback_budget_gate.json`, `logs/taskitem_runs/TEST_ONLY_sprint238_native_gate_fail_20260226.json`, `logs/taskitem_runs/TEST_ONLY_sprint239_semantic_fallback_audit_20260226/semantic_fallback_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint240_retry_20260226_145631/00_summary.json`, `docs/sprint236_execution_tracker_2026-02-26.md`, `docs/sprint237_execution_tracker_2026-02-26.md`, `docs/sprint238_execution_tracker_2026-02-26.md`, `docs/sprint239_execution_tracker_2026-02-26.md`, `docs/sprint240_execution_tracker_2026-02-26.md` | `partial` | Sprint 236 added deterministic fallback-gap auditing with tool + constraint metadata. Sprint 237 added fallback-budget hard gate (`max-fallback-rate`) and produced expected fail/pass artifacts. Sprint 238 added native decomposition hard gate in pipeline (`min task count`, `min semantic signal count`) so weak native generation can be blocked before fallback masking. Sprint 239 added native reason enrichment and improved semantic signal density (`0 -> 6`). Sprint 240 added native decomposition retry with explicit minimum-task policy; on current sample retry attempted but did not improve depth (`2 -> 2`). | Native decomposition task-depth upgrade (increase native task granularity beyond 2) |
|
||||
| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json` | `partial` | Sprint 241 added one-by-one impact profiles and a pipeline-integrated native impact coverage gate (`native_impact_coverage`). Sprint 242 added profile-driven remediation synthesis and rerun loop. Current hard fullstack sample remains unchanged after one remediation pass (`failing_profile_count 7 -> 7`), so capability closure is still pending. | Implement profile-specific decomposition operators that synthesize concrete task bundles, not only constraints |
|
||||
| GR-021 | Impact-specific native decomposition coverage not enforced uniformly | `docs/native_decomposition_impact_list_2026-02-26.md`, `tools/mcp/profiles/native_decomposition_impact_profiles.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_fullstack_impact_20260226_150416/00_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint241_native_impact_aggregate_20260226/native_impact_coverage_aggregate.json`, `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/remediation_loop_summary.json`, `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json` | `partial` | Sprint 241 added one-by-one impact profiles and a pipeline-integrated native impact coverage gate (`native_impact_coverage`). Sprint 242 added profile-driven remediation synthesis and rerun loop. Sprint 243 added profile-specific remediation task bundle synthesis + injection and updated coverage checking over effective task set; hard fullstack sample now closes in one rerun (`failing_profile_count 7 -> 0`). Native first-pass generator depth remains weak, so closure currently depends on remediation injection path. | Improve first-pass native generator decomposition so coverage passes without remediation injection |
|
||||
|
||||
## What Was Covered Today (Sprints 175-184)
|
||||
|
||||
|
||||
@@ -218,3 +218,19 @@
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint242_impact_remediation_loop_20260226/extra_normalized_requirements.json`
|
||||
- Current measured loop result on hard fullstack sample:
|
||||
- `failing_profile_count 7 -> 7` (wiring complete, capability uplift still pending)
|
||||
|
||||
## Sprint 243 Added (Same Day)
|
||||
|
||||
- Added profile-specific remediation task synthesis:
|
||||
- `tools/mcp/synthesize_native_impact_remediation_tasks.py`
|
||||
- Pipeline now supports injected extra tasks:
|
||||
- `WSTONE_EXTRA_TASKS_FILE`
|
||||
- summary packet: `extra_tasks`
|
||||
- Coverage checker now evaluates effective tasks (`native + injected`):
|
||||
- `tools/mcp/check_native_decomposition_impact_coverage.py`
|
||||
- Enhanced remediation loop injects both constraints and tasks:
|
||||
- `tools/mcp/run_native_impact_remediation_loop.sh`
|
||||
- Dated closure artifact:
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`
|
||||
- Current measured loop result on hard fullstack sample:
|
||||
- `failing_profile_count 7 -> 0`
|
||||
|
||||
39
docs/sprint243_execution_tracker_2026-02-26.md
Normal file
39
docs/sprint243_execution_tracker_2026-02-26.md
Normal file
@@ -0,0 +1,39 @@
|
||||
# Sprint 243 Execution Tracker - 2026-02-26
|
||||
|
||||
## Scope
|
||||
- `sprint243_plan.md`
|
||||
|
||||
## Implemented
|
||||
|
||||
Pipeline update in `tools/mcp/run_sprint_taskitem_pipeline.sh`:
|
||||
- Added `WSTONE_EXTRA_TASKS_FILE`.
|
||||
- Validates file exists and is JSON array.
|
||||
- Appends `extra_tasks` into effective task set before queue/validation.
|
||||
- Summary now includes `extra_tasks`.
|
||||
|
||||
New remediation task synthesizer:
|
||||
- `tools/mcp/synthesize_native_impact_remediation_tasks.py`
|
||||
- Creates one deterministic remediation task per failing impact profile with required ops/reasons/contracts.
|
||||
|
||||
Remediation loop enhancement:
|
||||
- `tools/mcp/run_native_impact_remediation_loop.sh`
|
||||
- now injects both:
|
||||
- `extra_normalized_requirements.json`
|
||||
- `extra_tasks.json`
|
||||
|
||||
Coverage checker update:
|
||||
- `tools/mcp/check_native_decomposition_impact_coverage.py`
|
||||
- evaluates effective tasks as `native_generated_tasks + extra_tasks`.
|
||||
|
||||
## Baseline Validation Artifact
|
||||
|
||||
- `logs/taskitem_runs/TEST_ONLY_sprint243_impact_remediation_tasks_loop_20260226_r2/remediation_loop_summary.json`
|
||||
|
||||
Observed result on hard fullstack sample:
|
||||
- before: `failing_profile_count=7`
|
||||
- after: `failing_profile_count=0`
|
||||
- delta: `-7`
|
||||
|
||||
## Explicit Completion Signal
|
||||
|
||||
- Sprint 243: `DONE` (implemented + rerun loop verified with profile-failure closure)
|
||||
Reference in New Issue
Block a user