whetstone_RSA/docs/architecture.md

Architecture

Purpose

whetstone_RSA should be a callable library for bounded fuzzy decisions, not a general chat model.

The reusable unit is not "a small transformer." The reusable unit is:

• a finite decision space
• a compact policy over that space
• a deterministic executor behind it

Core Runtime

The library should expose one high-level operation:

decide(input, schema, context) -> decision_result

Where:

• input is natural language or a compact structured description
• schema defines the legal decision space
• context carries app-specific state

The runtime returns:

• decision_id
• slots
• confidence
• abstain
• escalation_target
• trace

Main Components

1. Decision Schema

Defines the bounded output surface:

• enum labels
• slot definitions
• required vs optional fields
• validity constraints
• allowed escalation targets

This is the contract that keeps the model narrow.

2. Input Encoder

Normalizes app inputs into a stable inference shape.

Examples:

• plain user text
• AST node metadata
• taskitem summaries
• form descriptions
• UI state plus text

3. Policy Layer

A tiny learned model that scores legal outcomes.

This may start as a transformer specialist, but the interface should not hard-code one model family. The policy layer should be swappable.

4. Constraint Layer

Deterministic checks that reject illegal or inconsistent outputs.

Examples:

• invalid enum value
• missing required slot
• impossible field combination
• out-of-policy tool choice

5. Executor

Takes a valid decision and calls the deterministic downstream system:

• tool invocation
• template expansion
• AST mutation selection
• workflow dispatch
• form submission preparation

6. Confidence and Escalation

Every decision gate needs a native abstain path.

Outputs should support:

• proceed
• retry_with_context
• escalate_to_larger_model
• escalate_to_human

7. Gate Diagnosis

The runtime should eventually support gate diagnosis as a first-class library capability.

This means the library should not only run gates, but also help determine why a gate is underperforming and what restructuring options are plausible.

See docs/gate_failure_taxonomy.md.

Library Boundaries

The core library should stay domain-agnostic.

Recommended split:

• rsa-core: schemas, inference interface, scoring, confidence, tracing
• rsa-runtime: model loading, quantization support, inference backends
• rsa-eval: benchmarks, calibration, error analysis
• rsa-domains-whetstone: WhetstoneDSL gate schemas and adapters

Data Model

Each gate should be described by metadata that is independent of any one model:

• gate_id
• task_family
• output_cardinality
• slot_schema
• input_modalities
• requires_world_knowledge
• requires_stateful_context
• deterministic_fallback_available
• label_stability
• candidate_factorizations
• quality_target
• latency_budget_ms

This metadata matters because model size should be chosen from gate properties, not from intuition alone.

Initial API Sketch

GateDefinition
  id
  labels
  slots
  constraints
  escalation_policy

DecisionRequest
  gate_id
  input_text
  context

DecisionResult
  decision_id
  slots
  confidence
  abstain
  escalation_target
  trace

GateDiagnosis
  primary_failure_class
  secondary_failure_classes
  recommended_probes
  recommended_interventions
  confidence

Non-Goals

• open-ended answer generation
• internet-grounded knowledge retrieval
• replacing deterministic business logic
• hiding uncertainty instead of exposing it

Guiding Principle

Intelligence should be concentrated at the routing boundary.

Once the route is chosen and validated, the rest of the system should become ordinary software.