37 lines
1.2 KiB
Markdown
37 lines
1.2 KiB
Markdown
|
# Sprint 105 Plan: Benchmark-Driven Auto-Tuning of Policies
|
||
|
|
|
||
|
|
## Context
|
||
|
|
|
||
|
|
Policy defaults should adapt from measured outcomes. Sprint 105 adds benchmark-driven
|
||
|
|
auto-tuning proposals with strict guardrails.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Goals
|
||
|
|
|
||
|
|
1. Learn candidate policy parameter improvements from benchmark outcomes
|
||
|
|
2. Validate tuned policies in controlled replay environments
|
||
|
|
3. Compare tuned vs baseline policy packs objectively
|
||
|
|
4. Promote only if quality/risk constraints are met
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Steps
|
||
|
|
|
||
|
|
### Step 1279: Policy parameterization schema v2 (12 tests)
|
||
|
|
### Step 1280: Auto-tuning proposal generator from benchmark deltas (10 tests)
|
||
|
|
### Step 1281: Controlled policy replay harness (10 tests)
|
||
|
|
### Step 1282: Baseline-vs-tuned comparator with confidence bounds (10 tests)
|
||
|
|
### Step 1283: Tuning risk classifier and guardrail gates (10 tests)
|
||
|
|
### Step 1284: Promotion policy for tuned packs (8 tests)
|
||
|
|
### Step 1285: `whetstone_propose_policy_tuning` MCP tool (8 tests)
|
||
|
|
### Step 1286: `whetstone_validate_policy_tuning` MCP tool (8 tests)
|
||
|
|
### Step 1287: Policy tuning report artifact (8 tests)
|
||
|
|
### Step 1288: Sprint 105 integration summary + regression (8 tests)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Tuning Rule
|
||
|
|
|
||
|
|
- Tuned policies must beat baseline without increasing blocked-risk incidents.
|