Implement sprint 235 native-first semantic expansion policy

2026-02-26 14:50:32 -07:00
parent 7965899771
commit 3de32bc3d1
6 changed files with 111 additions and 3 deletions
--- a/docs/constructive_editing_runtime_plan.md
+++ b/docs/constructive_editing_runtime_plan.md
@@ -148,3 +148,10 @@ Parallel planning tranche (active):
 - baseline artifacts:
  - `logs/taskitem_runs/TEST_ONLY_spec_planning_baseline_20260226/fullstack_spec_readiness.json`
  - `logs/taskitem_runs/TEST_ONLY_spec_planning_baseline_20260226/deterministic_spec_readiness.json`
+
+Planning runtime controls (active):
+- `sprint232_plan.md` to `sprint235_plan.md`
+- semantic bridge + intake augmentation + requirement injection + expansion gating are wired into:
+  - `tools/mcp/run_sprint_taskitem_pipeline.sh`
+- current policy default is native-first semantic fallback:
+  - `WSTONE_SEMANTIC_TASK_EXPANSION_MODE=fallback_only`
--- a/docs/progress_log_2026-02-26.md
+++ b/docs/progress_log_2026-02-26.md
@@ -105,3 +105,21 @@
 - Integrated optional semantic gate into pipeline with hard-fail behavior.
 - Smoke-verified gate-enabled run (TEST_ONLY):
  - `logs/taskitem_runs/TEST_ONLY_sprint234_semantic_gate_smoke_20260226/00_summary.json`
+
+## Sprint 235 Added (Same Day)
+
+- Added native-first semantic expansion policy in pipeline:
+  - `WSTONE_SEMANTIC_TASK_EXPANSION_MODE` (`fallback_only` or `always`)
+- Added native decomposition telemetry:
+  - `native_task_count`
+  - `native_semantic_signal_count`
+- Added explicit `semantic_task_expansion` summary states:
+  - enabled + fallback applied
+  - enabled + skipped (native sufficient)
+  - disabled by configuration
+- Smoke-verified TEST_ONLY runs:
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_policy_20260226_145007/00_summary.json`
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_policy_20260226_145008/00_summary.json`
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_probe_20260226_144846/00_summary.json`
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_probe_20260226_144854/00_summary.json`
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_richspec_20260226_144902/00_summary.json`
--- a/docs/sprint235_execution_tracker_2026-02-26.md
+++ b/docs/sprint235_execution_tracker_2026-02-26.md
@@ -0,0 +1,34 @@
+# Sprint 235 Execution Tracker - 2026-02-26
+
+## Scope
+- `sprint235_plan.md`
+
+## Implemented
+
+Pipeline updates in `tools/mcp/run_sprint_taskitem_pipeline.sh`:
+- Added `WSTONE_SEMANTIC_TASK_EXPANSION_MODE` (default `fallback_only`).
+- Added native decomposition telemetry before semantic expansion:
+  - `native_task_count`
+  - `native_semantic_signal_count`
+- Added fallback policy:
+  - `always`: always apply semantic expansion.
+  - `fallback_only`: apply only when native decomposition is shallow.
+- Added explicit summary semantics for `semantic_task_expansion`:
+  - `enabled=true, fallback_applied=true` when semantic fallback expands tasks.
+  - `enabled=true, fallback_applied=false, skipped_reason=...` when enabled but skipped.
+  - `enabled=false, skipped_reason=semantic_expansion_disabled` when feature toggled off.
+
+## Baseline Smoke
+
+- fallback-only policy run (TEST_ONLY):
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_policy_20260226_145007/00_summary.json`
+- semantic expansion disabled-state run (TEST_ONLY):
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_policy_20260226_145008/00_summary.json`
+- prior mode comparison and rich-spec probes (TEST_ONLY):
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_probe_20260226_144846/00_summary.json`
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_probe_20260226_144854/00_summary.json`
+  - `logs/taskitem_runs/TEST_ONLY_sprint235_richspec_20260226_144902/00_summary.json`
+
+## Explicit Completion Signal
+
+- Sprint 235: `DONE` (implemented + integrated + smoke verified)
--- a/editor/src/Sprint235IntegrationSummary.h
+++ b/editor/src/Sprint235IntegrationSummary.h
@@ -0,0 +1,6 @@
+#pragma once
+
+// Sprint 235 integration summary:
+// - Added native-first semantic task expansion policy with mode selection.
+// - Pipeline now records native decomposition signals and explicit fallback decisions.
+// - Summary telemetry distinguishes disabled expansion from enabled-but-skipped execution.
--- a/sprint235_plan.md
+++ b/sprint235_plan.md
@@ -0,0 +1,12 @@
+# Sprint 235 Plan: Native-First Semantic Expansion Policy
+
+## Goal
+Prevent semantic task expansion from overriding already-strong native decomposition while preserving deterministic fallback for underspecified outputs.
+
+## Steps
+- Step 2217: Add semantic expansion policy mode (`always` vs `fallback_only`).
+- Step 2218: Compute native decomposition signals before semantic expansion.
+- Step 2219: Apply expansion only when native decomposition is too shallow under `fallback_only`.
+- Step 2220: Emit expansion telemetry with fallback decision metadata.
+- Step 2221: Distinguish "feature disabled" from "enabled but skipped" in summary output.
+- Step 2222: Add `Sprint235IntegrationSummary.h` and execution tracker.
--- a/tools/mcp/run_sprint_taskitem_pipeline.sh
+++ b/tools/mcp/run_sprint_taskitem_pipeline.sh
@@ -28,6 +28,7 @@ SEMANTIC_PLANNING_BRIDGE="${WSTONE_SEMANTIC_PLANNING_BRIDGE:-1}"
 SEMANTIC_INTAKE_AUGMENT="${WSTONE_SEMANTIC_INTAKE_AUGMENT:-1}"
 SEMANTIC_REQUIREMENT_INJECTION="${WSTONE_SEMANTIC_REQUIREMENT_INJECTION:-1}"
 SEMANTIC_TASK_EXPANSION="${WSTONE_SEMANTIC_TASK_EXPANSION:-1}"
+SEMANTIC_TASK_EXPANSION_MODE="${WSTONE_SEMANTIC_TASK_EXPANSION_MODE:-fallback_only}"
 SEMANTIC_COVERAGE_GATE="${WSTONE_SEMANTIC_COVERAGE_GATE:-0}"
 SEMANTIC_MIN_COVERAGE="${WSTONE_SEMANTIC_MIN_COVERAGE:-0.9}"
 SEMANTIC_REQUIRE_CAPS_FOR_COMPLEX="${WSTONE_SEMANTIC_REQUIRE_CAPS_FOR_COMPLEX:-1}"
@@ -285,7 +286,20 @@ if [[ "$(printf '%s' "$GEN_JSON" | jq -r '.success // false')" != "true" ]]; the
 fi

 TASKS="$(printf '%s' "$GEN_JSON" | jq '.tasks')"
-if [[ "$SEMANTIC_PLANNING_BRIDGE" == "1" && "$SEMANTIC_REQUIREMENT_INJECTION" == "1" && "$SEMANTIC_TASK_EXPANSION" == "1" && -f "$OUT_DIR/01c_semantic_injected_requirements.json" ]]; then
+native_task_count="$(printf '%s' "$TASKS" | jq 'length')"
+native_semantic_signal_count="$(printf '%s' "$TASKS" | jq '[.[] | ((.reasons // [])[]? | tostring) | select(test("semantic|risk|contract|capability"; "i"))] | length')"
+semantic_fallback_needed="false"
+if [[ "$SEMANTIC_TASK_EXPANSION_MODE" == "always" ]]; then
+  semantic_fallback_needed="true"
+elif [[ "$SEMANTIC_TASK_EXPANSION_MODE" == "fallback_only" ]]; then
+  if [[ "$native_task_count" -le 2 ]]; then
+    semantic_fallback_needed="true"
+  elif [[ "$native_task_count" -le 4 && "$native_semantic_signal_count" -eq 0 ]]; then
+    semantic_fallback_needed="true"
+  fi
+fi
+
+if [[ "$SEMANTIC_PLANNING_BRIDGE" == "1" && "$SEMANTIC_REQUIREMENT_INJECTION" == "1" && "$SEMANTIC_TASK_EXPANSION" == "1" && "$semantic_fallback_needed" == "true" && -f "$OUT_DIR/01c_semantic_injected_requirements.json" ]]; then
  SEMANTIC_EXPANDED_TASKS="$(jq -nc \
    --argjson base "$TASKS" \
    --argjson sem "$(cat "$OUT_DIR/01c_semantic_injected_requirements.json")" '
@@ -312,9 +326,26 @@ if [[ "$SEMANTIC_PLANNING_BRIDGE" == "1" && "$SEMANTIC_REQUIREMENT_INJECTION" ==
      | ($base + $extra)
    ')"
  TASKS="$SEMANTIC_EXPANDED_TASKS"
-  SEMANTIC_TASK_EXPANSION_JSON="$(jq -nc --argjson sem "$(cat "$OUT_DIR/01c_semantic_injected_requirements.json")" '{enabled:true, expanded_task_count: (($sem[:6])|length)}')"
+  SEMANTIC_TASK_EXPANSION_JSON="$(jq -nc \
+    --argjson sem "$(cat "$OUT_DIR/01c_semantic_injected_requirements.json")" \
+    --arg mode "$SEMANTIC_TASK_EXPANSION_MODE" \
+    --argjson native_task_count "$native_task_count" \
+    --argjson native_semantic_signal_count "$native_semantic_signal_count" \
+    '{enabled:true, mode:$mode, fallback_applied:true, expanded_task_count: (($sem[:6])|length), native_task_count:$native_task_count, native_semantic_signal_count:$native_semantic_signal_count}')"
 else
-  SEMANTIC_TASK_EXPANSION_JSON='{"enabled":false,"expanded_task_count":0}'
+  if [[ "$SEMANTIC_PLANNING_BRIDGE" == "1" && "$SEMANTIC_REQUIREMENT_INJECTION" == "1" && "$SEMANTIC_TASK_EXPANSION" == "1" ]]; then
+    SEMANTIC_TASK_EXPANSION_JSON="$(jq -nc \
+      --arg mode "$SEMANTIC_TASK_EXPANSION_MODE" \
+      --argjson native_task_count "$native_task_count" \
+      --argjson native_semantic_signal_count "$native_semantic_signal_count" \
+      '{enabled:true, mode:$mode, fallback_applied:false, expanded_task_count:0, native_task_count:$native_task_count, native_semantic_signal_count:$native_semantic_signal_count, skipped_reason:"native_sufficient_or_missing_semantic_requirements"}')"
+  else
+    SEMANTIC_TASK_EXPANSION_JSON="$(jq -nc \
+      --arg mode "$SEMANTIC_TASK_EXPANSION_MODE" \
+      --argjson native_task_count "$native_task_count" \
+      --argjson native_semantic_signal_count "$native_semantic_signal_count" \
+      '{enabled:false, mode:$mode, fallback_applied:false, expanded_task_count:0, native_task_count:$native_task_count, native_semantic_signal_count:$native_semantic_signal_count, skipped_reason:"semantic_expansion_disabled"}')"
+  fi
 fi
 QUEUE_ARGS="$(jq -nc --argjson t "$TASKS" --argjson nr "$NORMALIZED_REQS" --arg strict "$STRICT_EXECUTION_CONTRACT" --argjson cs "$CAPABILITY_SIGNALS_JSON" \
  '{tasks:$t,normalizedRequirements:$nr,strictExecutionContract:($strict == "1"),capabilitySignals:$cs}')"