whetstone_RSA/ROADMAP.md

Roadmap

Phase 0: Definition

• Write the project framing and library boundaries.
• Define the bounded gate schema contract.
• Document the WhetstoneDSL case study and current specialist fleet.

Phase 0A: AST-First Realignment

Before whetstone_RSA grows much further as a hand-authored C++ library, realign the project so its semantic source of truth is Whetstone-authored and C++ becomes a projection target.

This phase exists to prevent the current bootstrap scaffold from hardening into the wrong long-term architecture.

Intent

• make RSA semantically AST-first
• keep C++ as runtime projection and execution layer
• preserve compatibility with the current scaffold while changing the source of truth

Architectural rule

The long-term source of truth for RSA should live in Whetstone semantic forms, not in hand-authored C++ structs.

The C++ code in this repository should be treated as:

• runtime implementation
• projected contract types
• backend execution surface
• diagnosis and policy engine

not the canonical definition of RSA semantics.

0A.1 Semantic entities to define first

Define Whetstone-authored semantic objects for at least:

• GateDefinition
• DecisionRequest
• DecisionResult
• StructuredDecisionResult
• GateSuitability
• GateDiagnosis
• ProbeDefinition
• BackendSuitability
• GatePackage

These should carry the contract metadata already identified in the docs:

• input structure and modality
• output topology and constraint
• policy stability
• gate shape
• confidence, abstain, and escalation policy
• trace fields
• backend family and packaging metadata

0A.2 Projection boundary

Define a deliberate projection boundary from Whetstone semantic forms into:

• C++ runtime contract structs
• schema artifacts
• benchmark configuration artifacts
• packaged runtime manifests

The important architectural constraint is:

• semantics are authored once
• runtime targets are projected from that source

0A.3 Migration rule for the current scaffold

Do not continue expanding hand-authored C++ contract definitions unless they are clearly temporary runtime placeholders.

When a contract type changes, the preferred direction should become:

• update semantic source
• regenerate or realign projected C++ representation

rather than:

• evolve C++ first
• treat docs and later AST work as catch-up

0A.4 Immediate AST-first deliverables

• define the first RSA semantic model in Whetstone terms
• map current C++ runtime types to those semantic entities
• identify what current fields are semantic contract versus runtime-only detail
• define the projection rules into C++
• update roadmap phases so future implementation assumes semantic-first authoring

0A.5 Scope constraint

This does not require immediate deep modification of whetstone_DSL runtime code.

It does require that future RSA design work stop assuming C++ is the authoring home of the system.

The immediate goal is architectural realignment, not premature full conversion.

Phase 1: Gate Inventory

• Enumerate candidate gates from WhetstoneDSL.
• Record output vocabulary, input shape, and downstream deterministic executor.
• Split gates into deterministic, specialist, and escalation-first candidates.

Primary case-study inventory artifact:

• docs/case_studies/whetstone_dsl_gate_inventory.md

Phase 2: Entropy Baseline

• Define a first-pass gate entropy score.
• Score the initial WhetstoneDSL gate set.
• Identify which gates are plausible at the ~800KB tier and which are not.

Phase 3: Size Sweep

• Run fixed-data comparisons across multiple model sizes.
• Measure accuracy, calibration, confusion concentration, and latency.
• Produce the first entropy-to-tier baseline table.

Phase 4: Core Library

• Implement gate schema loading and validation.
• Implement the decision request/response runtime.
• Implement abstain and escalation policies.
• Support pluggable inference backends.

Phase 4A: Post-Suitability RSA Runtime Plan

This phase begins only after a decision surface has already been judged to belong in the rsa layer rather than deterministic, slm, llm, or human.

The focus here is not broad suitability triage. The focus is:

• how an RSA gate is represented
• how its shape is chosen
• how its backend is selected
• how it is packaged for efficient runtime use

Design constraints

• Keep the library focused on whetstone_RSA, not direct whetstone_DSL changes.
• Assume RSA semantic contracts are authored AST-first and projected into runtime targets rather than defined primarily in handwritten C++.
• Support both bounded single-choice classification and bounded structured outputs with slots from the beginning.
• Include rl_policy and bandit-style policies as contract placeholders, but do not make them a primary implementation target until classical ML and transformer backends are working and benchmarked.
• Treat hybrid inputs as first-class. A gate may consume text, symbolic state, and numeric features together.
• Use case studies only to surface reusable abstractions. Do not let one case study become the semantic definition of the library.

4A.1 Contract and packet model

• Define a canonical GateDefinition that supports:
  • bounded enum decisions
  • bounded structured slot-filling decisions
  • constraints
  • confidence policy
  • abstain policy
  • escalation policy
  • trace metadata
• Define a canonical DecisionRequest packet that supports:
  • textual fields
  • symbolic fields
  • numeric features
  • state and provenance metadata
• Define DecisionResult and StructuredDecisionResult so programs can consume either plain label outputs or bounded slot outputs through argument-oriented runtime calls rather than project-specific ad hoc interfaces.

4A.2 Gate-shape selection

The library should not assume that all bounded problems should be represented as one flat multiclass classifier.

For each RSA candidate gate, support shape analysis across:

• binary
• multiclass
• factorized
• hierarchical

Initial implementation goals:

• define metadata that describes candidate gate shapes
• define diagnostics that detect when a flat label space hides compositional structure
• support shape-specific constraints so invalid state combinations are prevented by contract rather than patched later

4A.3 Backend-selection layer

Backend selection remains separate from RSA identity.

The library should support a backend contract for at least:

• linear_model
• gradient_boosting
• random_forest
• bayesian
• transformer
• hybrid
• rl_policy placeholder

Initial implementation goals:

• define a common backend interface for train, infer, calibrate, and trace
• allow external benchmark or AutoML-style workflows to produce backend recommendations without forcing one internal training stack
• keep backend artifacts swappable as long as the gate contract is preserved

4A.4 Benchmark harness

Build a reusable harness that compares both gate shape and backend family for the same bounded decision contract.

For each gate, the harness should compare:

• flat multiclass against factorized or hierarchical alternatives
• classical ML backends against tiny transformer backends
• deterministic baselines where any part of the gate can be rule-complete

Required outputs:

• accepted-decision accuracy
• calibration quality
• abstain quality
• invalid-output rate
• downstream deterministic validation success
• latency
• memory footprint
• training and retraining cost

4A.5 Runtime packaging

AutoML or scripted comparison should be offline or occasional, not on the hot path.

The runtime should load a selected gate package that includes:

• gate contract
• chosen gate shape
• chosen backend family
• trained artifact reference
• confidence thresholds
• abstain and escalation policy
• trace schema and version metadata

The steady-state runtime model is:

• benchmark and select offline
• package once
• load the chosen gate into memory
• serve repeated decisions efficiently
• re-run selection only when schema, data, policy, or hardware conditions change

4A.6 Invocation surface

The library should expose explicit runtime calls rather than force consumers to rebuild invocation conventions for each gate.

Initial target surface:

• decide_class(request, gate_definition)
• decide_structured(request, gate_definition)
• diagnose_gate(gate_definition, evidence, benchmark_results)
• recommend_backend(gate_definition, benchmark_results)
• recommend_gate_shape(gate_definition, benchmark_results)

The exact function names may change, but the core idea should remain: consumers pass structured arguments into a reusable RSA runtime surface.

4A.7 Diagnosis loop

When a gate underperforms, the first response should not be automatic model-size escalation.

The runtime and benchmark surfaces should instead help answer:

• should this gate stay flat or be factorized?
• is part of the task actually deterministic?
• is the label space unstable?
• is the backend family mismatched to the input packet?
• should the gate abstain earlier?

4A.8 First case-study role

Use the memory-strategy decision only as the first end-to-end exercise for the abstract mechanism.

The case study should help validate:

• mixed symbolic/text/numeric packet design
• shape comparison between flat and decomposed gates
• backend comparison between classical ML and tiny transformers
• packaging of both enum-style and slot-filling outputs

It should not be allowed to overfit the library to one memory-management ontology.

4A.9 Immediate implementation order

• define the AST-first semantic source for RSA contracts
• define projection rules from semantic entities into C++ runtime types
• extend schema types so both enum and structured-slot gates are first-class
• define canonical request/result packet types
• define gate-shape metadata and recommendation types
• define backend-family contracts, including placeholder RL/bandit support
• build the benchmark report format
• implement classical ML adapters first
• implement transformer adapter second
• add packaging and runtime loading
• add diagnosis and recommendation APIs on top of benchmark evidence

Phase 5: Whetstone Integration

• Expose WhetstoneDSL gates through the library.
• Replace selected MCP bounded decisions with local specialist calls.
• Log abstentions and misroutes for retraining and gate redesign.

Phase 6: Hivemind Integration

• Feed gate failure data into Hivemind as entropy evidence.
• Use Hivemind to prioritize which high-entropy gates deserve deterministic tooling.
• Distinguish "needs a bigger model" from "needs a better tool or schema."

Exit Criteria For Version 0

• At least one reusable gate schema format is stable.
• Both enum-style and bounded structured-slot gates are supported by the contract.
• At least three WhetstoneDSL gates are benchmarked across size tiers.
• At least one gate has been compared across multiple gate shapes, not just multiple model sizes.
• At least one gate has been compared across classical ML and transformer backends under the same contract.
• A baseline entropy-to-tier recommendation table exists.
• One real application can call the library instead of hard-coding model selection or backend choice.