Sprint 003 findings + Sprint 004 pipeline decision audit plan

- split_prereq_op_dataset.py: extract accepted/rejected/schema-drift rows per gate
- gen_prereq_op_rsa_data.py: convert accepted rows to Fabricate TSV format
- 1000-run extracted dataset: 1297 accepted prereq_op rows, 24 schema-drift
- Key finding: needs_validate_intake always True; effective gate is binary
  (needs_resolve_dependencies 77%, needs_architect_review 20%)
- Key finding: run corpus text is template-driven (17 unique inputs) — not
  suitable for text-based training; RSA exists before ideal input is available
- ROADMAP: gate lifecycle framing — goal is choice of backend, not forced
  determinism; RSA operates on imperfect/mixed/incomplete input by design
- training_data_strategy.md: taskitem-as-self-contained-work-unit constraint
- SPRINT-003: updated with 1000-run results, text audit findings, design constraints
- SPRINT-004: pipeline decision audit — manual walkthrough of every decision
  from project description to generated artifact; execute next session

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bill Holcombe
2026-04-12 21:43:17 -06:00
parent 0253a92f28
commit c4569de9ca
18 changed files with 10209 additions and 4 deletions

View File

@@ -322,17 +322,33 @@ It should not be allowed to overfit the library to one memory-management ontolog
- add packaging and runtime loading
- add diagnosis and recommendation APIs on top of benchmark evidence
## Gate Lifecycle — The Fundamental Property
RSA gates exist **before ideal input is available**. The intake may return a mix of
structured and unstructured data, or incomplete directions. That is the design
requirement, not a defect to fix upstream.
The RSA layer gives operators **choice of backend** per gate: statistical model,
deterministic rules, SLM, LLM, or human escalation. As the contract matures, more
backend options become viable. Some gates may remain statistical models indefinitely
because that is the right fit. Others may be replaced by rules. The operator decides.
Abstain and escalation policies are first-class responses to weak input — not failure
modes. A gate that correctly abstains on incomplete input is working as designed.
## Phase 5: Whetstone Integration
- Expose WhetstoneDSL gates through the library.
- Replace selected MCP bounded decisions with local specialist calls.
- Log abstentions and misroutes for retraining and gate redesign.
- For each deployed gate: track which backend is active and what would justify switching.
## Phase 6: Hivemind Integration
- Feed gate failure data into Hivemind as entropy evidence.
- Use Hivemind to prioritize which high-entropy gates deserve deterministic tooling.
- Distinguish "needs a bigger model" from "needs a better tool or schema."
- Support backend substitution per gate without changing the gate contract.
## Exit Criteria For Version 0