Sprint 003 findings + Sprint 004 pipeline decision audit plan
- split_prereq_op_dataset.py: extract accepted/rejected/schema-drift rows per gate - gen_prereq_op_rsa_data.py: convert accepted rows to Fabricate TSV format - 1000-run extracted dataset: 1297 accepted prereq_op rows, 24 schema-drift - Key finding: needs_validate_intake always True; effective gate is binary (needs_resolve_dependencies 77%, needs_architect_review 20%) - Key finding: run corpus text is template-driven (17 unique inputs) — not suitable for text-based training; RSA exists before ideal input is available - ROADMAP: gate lifecycle framing — goal is choice of backend, not forced determinism; RSA operates on imperfect/mixed/incomplete input by design - training_data_strategy.md: taskitem-as-self-contained-work-unit constraint - SPRINT-003: updated with 1000-run results, text audit findings, design constraints - SPRINT-004: pipeline decision audit — manual walkthrough of every decision from project description to generated artifact; execute next session Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
16
ROADMAP.md
16
ROADMAP.md
@@ -322,17 +322,33 @@ It should not be allowed to overfit the library to one memory-management ontolog
|
||||
- add packaging and runtime loading
|
||||
- add diagnosis and recommendation APIs on top of benchmark evidence
|
||||
|
||||
## Gate Lifecycle — The Fundamental Property
|
||||
|
||||
RSA gates exist **before ideal input is available**. The intake may return a mix of
|
||||
structured and unstructured data, or incomplete directions. That is the design
|
||||
requirement, not a defect to fix upstream.
|
||||
|
||||
The RSA layer gives operators **choice of backend** per gate: statistical model,
|
||||
deterministic rules, SLM, LLM, or human escalation. As the contract matures, more
|
||||
backend options become viable. Some gates may remain statistical models indefinitely
|
||||
because that is the right fit. Others may be replaced by rules. The operator decides.
|
||||
|
||||
Abstain and escalation policies are first-class responses to weak input — not failure
|
||||
modes. A gate that correctly abstains on incomplete input is working as designed.
|
||||
|
||||
## Phase 5: Whetstone Integration
|
||||
|
||||
- Expose WhetstoneDSL gates through the library.
|
||||
- Replace selected MCP bounded decisions with local specialist calls.
|
||||
- Log abstentions and misroutes for retraining and gate redesign.
|
||||
- For each deployed gate: track which backend is active and what would justify switching.
|
||||
|
||||
## Phase 6: Hivemind Integration
|
||||
|
||||
- Feed gate failure data into Hivemind as entropy evidence.
|
||||
- Use Hivemind to prioritize which high-entropy gates deserve deterministic tooling.
|
||||
- Distinguish "needs a bigger model" from "needs a better tool or schema."
|
||||
- Support backend substitution per gate without changing the gate contract.
|
||||
|
||||
## Exit Criteria For Version 0
|
||||
|
||||
|
||||
Reference in New Issue
Block a user