whetstone_RSA/docs/backend_selection.md

# Backend Selection

## Purpose

This document defines backend selection as a first-class concern for
`whetstone_RSA`.

The key distinction is:

- RSA decides whether a problem should be handled as a bounded decision contract
- backend selection decides which implementation family should fill that contract

These are different questions and should not be collapsed together.

## Core Rule

Do not define RSA by one backend family.

RSA is model-agnostic.
The backend should be selected based on gate characteristics, deployment needs, and
measured results.

## Decision Layers

There are three separate decisions:

### 1. Layer Suitability

Question:

Should this problem be:

- `deterministic`
- `rsa`
- `slm`
- `llm`
- `human`

This is handled by suitability assessment.

### 2. Gate Shape Selection

Question:

If the problem is an RSA candidate, what shape should the gate take?

- `binary`
- `multiclass`
- `factorized`
- `hierarchical`

This is handled by gate design and diagnosis.

### 3. Backend Selection

Question:

If the problem is an RSA gate, what implementation family should fill it?

- `transformer`
- `gradient_boosting`
- `random_forest`
- `bayesian`
- `linear_model`
- `rl_policy`
- `hybrid`

This document is about the third decision.

## Why This Matters

Different gates have different data characteristics.

Examples:

- `unstructured + textual + bounded`
  - transformer may be the natural default

- `structured + numeric + bounded`
  - gradient boosting, random forest, or linear models may be stronger and cheaper

- `small data + strong priors`
  - Bayesian or simpler classifiers may be better

- `reward-shaped routing with delayed payoff`
  - RL or bandit-style policies may become useful

So backend choice should not be hard-coded.

## What RSA Should Own

`whetstone_RSA` should own:

- the gate contract
- suitability assessment
- diagnosis
- policy behavior
- the runtime interface for backends

It should not assume:

- transformer by default in all cases
- one training stack
- one benchmark methodology

## What Backend Selection Should Do

Backend selection should answer:

- which backend families are plausible for this gate?
- which should be tried first?
- which benchmark signals matter?
- whether existing model-selection tooling should be used

## Selection Variables

Backend recommendation should consider at least these variables:

### Contract variables

- `input_structure`
- `input_modality`
- `output_topology`
- `output_constraint`
- `policy_stability`

### Data variables

- dataset size
- class balance
- feature sparsity
- sequence length
- label noise
- factorization availability

### Deployment variables

- latency budget
- memory budget
- retraining frequency
- calibration needs
- explainability needs
- online adaptation needs

### Runtime variables

- deterministic fallback availability
- confidence thresholding support
- retry/escalation policy
- traceability requirements

## Suggested Types

Future implementation should likely include types along these lines:

```text
BackendFamily
  transformer
  gradient_boosting
  random_forest
  bayesian
  linear_model
  rl_policy
  hybrid
```

```text
BackendSuitability
  gate_id
  candidate_backends
  recommended_backend
  rationale
  confidence
```

```text
BackendBenchmarkResult
  gate_id
  backend_family
  metrics
  training_cost
  runtime_cost
  calibration
  notes
```

## Use Existing Tools Where Practical

Backend selection is not a place to reinvent the entire applied ML ecosystem.

In many cases, existing tools or benchmark harnesses can already compare:

- random forest
- boosted trees
- linear models
- Naive Bayes
- simple neural models

So `whetstone_RSA` should be willing to consume benchmark results from external
model-selection or AutoML-style workflows instead of owning every search strategy
itself.

The important thing is that the result plugs back into the RSA contract layer.

## Recommended Architectural Boundary

Keep backend selection adjacent to the RSA core, not inside the core identity.

That means:

- `GateSuitability` answers whether RSA is the right container
- `GateDiagnosis` helps debug and restructure the gate
- `BackendSuitability` recommends what implementation family should fill it

This separation keeps the architecture clean.

## Immediate Follow-Up

This should eventually be reflected in code by:

- adding a backend family enum
- adding backend suitability types
- extending the runtime contract so gates do not assume one backend
- optionally adding hooks for external benchmark/model-selection tooling

Until then, this document is the explicit placeholder so the work does not get lost.