whetstone_RSA/docs/architecture.md

# Architecture

## Purpose

`whetstone_RSA` should be a callable library for bounded fuzzy decisions, not a
general chat model.

The reusable unit is not "a small transformer." The reusable unit is:

- a finite decision space
- a compact policy over that space
- a deterministic executor behind it

## Core Runtime

The library should expose one high-level operation:

```text
decide(input, schema, context) -> decision_result
```

Where:

- `input` is natural language or a compact structured description
- `schema` defines the legal decision space
- `context` carries app-specific state

The runtime returns:

- `decision_id`
- `slots`
- `confidence`
- `abstain`
- `escalation_target`
- `trace`

## Main Components

### 1. Decision Schema

Defines the bounded output surface:

- enum labels
- slot definitions
- required vs optional fields
- validity constraints
- allowed escalation targets

This is the contract that keeps the model narrow.

### 2. Input Encoder

Normalizes app inputs into a stable inference shape.

Examples:

- plain user text
- AST node metadata
- taskitem summaries
- form descriptions
- UI state plus text

### 3. Policy Layer

A tiny learned model that scores legal outcomes.

This may start as a transformer specialist, but the interface should not hard-code
one model family. The policy layer should be swappable.

### 4. Constraint Layer

Deterministic checks that reject illegal or inconsistent outputs.

Examples:

- invalid enum value
- missing required slot
- impossible field combination
- out-of-policy tool choice

### 5. Executor

Takes a valid decision and calls the deterministic downstream system:

- tool invocation
- template expansion
- AST mutation selection
- workflow dispatch
- form submission preparation

### 6. Confidence and Escalation

Every decision gate needs a native abstain path.

Outputs should support:

- `proceed`
- `retry_with_context`
- `escalate_to_larger_model`
- `escalate_to_human`

### 7. Gate Diagnosis

The runtime should eventually support gate diagnosis as a first-class library
capability.

This means the library should not only run gates, but also help determine why a
gate is underperforming and what restructuring options are plausible.

See `docs/gate_failure_taxonomy.md`.

## Library Boundaries

The core library should stay domain-agnostic.

Recommended split:

- `rsa-core`: schemas, inference interface, scoring, confidence, tracing
- `rsa-runtime`: model loading, quantization support, inference backends
- `rsa-eval`: benchmarks, calibration, error analysis
- `rsa-domains-whetstone`: WhetstoneDSL gate schemas and adapters

## Data Model

Each gate should be described by metadata that is independent of any one model:

- `gate_id`
- `task_family`
- `output_cardinality`
- `slot_schema`
- `input_modalities`
- `requires_world_knowledge`
- `requires_stateful_context`
- `deterministic_fallback_available`
- `label_stability`
- `candidate_factorizations`
- `quality_target`
- `latency_budget_ms`

This metadata matters because model size should be chosen from gate properties,
not from intuition alone.

## Initial API Sketch

```text
GateDefinition
  id
  labels
  slots
  constraints
  escalation_policy

DecisionRequest
  gate_id
  input_text
  context

DecisionResult
  decision_id
  slots
  confidence
  abstain
  escalation_target
  trace

GateDiagnosis
  primary_failure_class
  secondary_failure_classes
  recommended_probes
  recommended_interventions
  confidence
```

## Non-Goals

- open-ended answer generation
- internet-grounded knowledge retrieval
- replacing deterministic business logic
- hiding uncertainty instead of exposing it

## Guiding Principle

Intelligence should be concentrated at the routing boundary.

Once the route is chosen and validated, the rest of the system should become
ordinary software.