105 lines
4.1 KiB
Markdown
105 lines
4.1 KiB
Markdown
|
# Sprint 163 Plan: Strict Grammar Foundation and Schema Normalization
|
||
|
|
|
||
|
|
## Context
|
||
|
|
|
||
|
|
Current MCP grammar coverage is complete (347 tools), but strictness is not maximal.
|
||
|
|
The grammar generator still falls back to broad productions (`any-object`,
|
||
|
|
`any-array`, `any-value`) for many nested argument shapes, which permits syntactic
|
||
|
|
validity without full schema-level correctness.
|
||
|
|
|
||
|
|
This sprint builds the strictness foundation:
|
||
|
|
|
||
|
|
1. define a measurable strictness score
|
||
|
|
2. normalize per-tool schemas into a canonical, recursion-safe form
|
||
|
|
3. refactor grammar generation so strict handling is possible for every schema node
|
||
|
|
|
||
|
|
No MCP tool names change. Tool contracts stay backward-compatible at the API level.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Goals
|
||
|
|
|
||
|
|
1. Add strictness metrics for all tool schemas and generated grammars
|
||
|
|
2. Introduce canonical schema normalization for recursive codegen
|
||
|
|
3. Eliminate fallback productions at top-level tool arguments
|
||
|
|
4. Keep generated artifacts deterministic across runs
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Steps
|
||
|
|
|
||
|
|
### Step 1859: Strictness audit utility and baseline report (10 tests)
|
||
|
|
|
||
|
|
Add `tools/mcp/audit_grammar_strictness.py`.
|
||
|
|
|
||
|
|
Input:
|
||
|
|
- `tools/mcp/whetstone_tool_schemas.json`
|
||
|
|
- `tools/mcp/grammars/per_tool_schemas.json`
|
||
|
|
- `tools/mcp/grammars/*.gbnf`
|
||
|
|
|
||
|
|
Output:
|
||
|
|
- `tools/mcp/grammars/strictness_report.json`
|
||
|
|
- per-tool metrics: fallback count, nested coverage, enum coverage, unsupported keywords
|
||
|
|
|
||
|
|
Tests (10): deterministic output, counts align with 347 tools, missing-file handling,
|
||
|
|
invalid schema handling, fallback detection, enum detection, nested object detection,
|
||
|
|
array-item strictness detection, summary rollup correctness, JSON schema for report.
|
||
|
|
|
||
|
|
### Step 1860: Canonical schema normalizer (12 tests)
|
||
|
|
|
||
|
|
Add `tools/mcp/schema_normalizer.py`.
|
||
|
|
|
||
|
|
Normalize each tool schema into a canonical AST-like structure:
|
||
|
|
- explicit node kinds: object, array, string, integer, number, boolean, null
|
||
|
|
- preserved constraints: required, additionalProperties, enum, const, pattern,
|
||
|
|
min/max, minItems/maxItems, oneOf/anyOf/allOf
|
||
|
|
- `$ref` expansion within document scope
|
||
|
|
|
||
|
|
Tests (12): nested object normalization, array-of-object normalization, enum/const
|
||
|
|
preservation, required/optional split, additionalProperties preservation, oneOf,
|
||
|
|
anyOf, allOf flattening semantics, `$ref` resolution, cycle guard, stable key order,
|
||
|
|
unsupported keyword capture, malformed schema fallback with diagnostics.
|
||
|
|
|
||
|
|
### Step 1861: Refactor grammar generator to use normalized schema IR (10 tests)
|
||
|
|
|
||
|
|
Update `tools/mcp/generate_tool_grammars.py` to consume normalized schema IR from
|
||
|
|
Step 1860 rather than ad-hoc field inspection.
|
||
|
|
|
||
|
|
No strict keyword expansion yet; this step is structural refactor only.
|
||
|
|
|
||
|
|
Tests (10): generated grammar parity for simple tools, deterministic ordering,
|
||
|
|
dispatch includes all tools, per-tool files still generated for all tools,
|
||
|
|
no regression in existing enum behavior, integer/number separation preserved,
|
||
|
|
boolean/null handling preserved, optional field handling preserved, required field
|
||
|
|
ordering stable, script exit code semantics unchanged.
|
||
|
|
|
||
|
|
### Step 1862: Top-level strictness gate (8 tests)
|
||
|
|
|
||
|
|
Disallow `any-object`/`any-array` at top-level argument properties unless explicitly
|
||
|
|
annotated with `x-whetstone-allow-broad: true` in schema metadata.
|
||
|
|
|
||
|
|
Tests (8): gate fails on broad top-level property, gate passes when annotated,
|
||
|
|
gate passes when strict object spec exists, failure message includes tool+field,
|
||
|
|
report includes waiver count, waivers are deterministic, unknown annotation ignored,
|
||
|
|
CI-friendly nonzero exit on violation.
|
||
|
|
|
||
|
|
### Step 1863: Sprint 163 Integration Summary (8 tests)
|
||
|
|
|
||
|
|
Add `editor/src/Sprint163IntegrationSummary.h`.
|
||
|
|
Record: steps_completed=5 (1859-1863), strictness_baseline_established=true,
|
||
|
|
top_level_broad_fallback_blocked=true, success=true.
|
||
|
|
|
||
|
|
Tests (8): constructable summary struct, steps_completed==5, baseline artifact exists,
|
||
|
|
normalizer available, generator refactor active, strictness gate active, no tool-count
|
||
|
|
regression (347), success==true.
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Architecture Gate
|
||
|
|
|
||
|
|
- No MCP runtime behavior changes; this sprint is tooling + generation pipeline only
|
||
|
|
- Generated grammar artifacts remain in `tools/mcp/grammars/`
|
||
|
|
- Deterministic generation is mandatory (stable ordering)
|
||
|
|
- All strictness checks must be machine-readable JSON
|
||
|
|
|