whetstone_RSA/docs/backend_selection.md

Backend Selection

Purpose

This document defines backend selection as a first-class concern for whetstone_RSA.

The key distinction is:

• RSA decides whether a problem should be handled as a bounded decision contract
• backend selection decides which implementation family should fill that contract

These are different questions and should not be collapsed together.

Core Rule

Do not define RSA by one backend family.

RSA is model-agnostic. The backend should be selected based on gate characteristics, deployment needs, and measured results.

Decision Layers

There are three separate decisions:

1. Layer Suitability

Question:

Should this problem be:

• deterministic
• rsa
• slm
• llm
• human

This is handled by suitability assessment.

2. Gate Shape Selection

Question:

If the problem is an RSA candidate, what shape should the gate take?

• binary
• multiclass
• factorized
• hierarchical

This is handled by gate design and diagnosis.

3. Backend Selection

Question:

If the problem is an RSA gate, what implementation family should fill it?

• transformer
• gradient_boosting
• random_forest
• bayesian
• linear_model
• rl_policy
• hybrid

This document is about the third decision.

Why This Matters

Different gates have different data characteristics.

Examples:

• unstructured + textual + bounded

  • transformer may be the natural default

• structured + numeric + bounded

  • gradient boosting, random forest, or linear models may be stronger and cheaper

• small data + strong priors

  • Bayesian or simpler classifiers may be better

• reward-shaped routing with delayed payoff

  • RL or bandit-style policies may become useful

So backend choice should not be hard-coded.

What RSA Should Own

whetstone_RSA should own:

• the gate contract
• suitability assessment
• diagnosis
• policy behavior
• the runtime interface for backends

It should not assume:

• transformer by default in all cases
• one training stack
• one benchmark methodology

What Backend Selection Should Do

Backend selection should answer:

• which backend families are plausible for this gate?
• which should be tried first?
• which benchmark signals matter?
• whether existing model-selection tooling should be used

Selection Variables

Backend recommendation should consider at least these variables:

Contract variables

• input_structure
• input_modality
• output_topology
• output_constraint
• policy_stability

Data variables

• dataset size
• class balance
• feature sparsity
• sequence length
• label noise
• factorization availability

Deployment variables

• latency budget
• memory budget
• retraining frequency
• calibration needs
• explainability needs
• online adaptation needs

Runtime variables

• deterministic fallback availability
• confidence thresholding support
• retry/escalation policy
• traceability requirements

Suggested Types

Future implementation should likely include types along these lines:

BackendFamily
  transformer
  gradient_boosting
  random_forest
  bayesian
  linear_model
  rl_policy
  hybrid

BackendSuitability
  gate_id
  candidate_backends
  recommended_backend
  rationale
  confidence

BackendBenchmarkResult
  gate_id
  backend_family
  metrics
  training_cost
  runtime_cost
  calibration
  notes

Use Existing Tools Where Practical

Backend selection is not a place to reinvent the entire applied ML ecosystem.

In many cases, existing tools or benchmark harnesses can already compare:

• random forest
• boosted trees
• linear models
• Naive Bayes
• simple neural models

So whetstone_RSA should be willing to consume benchmark results from external model-selection or AutoML-style workflows instead of owning every search strategy itself.

The important thing is that the result plugs back into the RSA contract layer.

Recommended Architectural Boundary

Keep backend selection adjacent to the RSA core, not inside the core identity.

That means:

• GateSuitability answers whether RSA is the right container
• GateDiagnosis helps debug and restructure the gate
• BackendSuitability recommends what implementation family should fill it

This separation keeps the architecture clean.

Immediate Follow-Up

This should eventually be reflected in code by:

• adding a backend family enum
• adding backend suitability types
• extending the runtime contract so gates do not assume one backend
• optionally adding hooks for external benchmark/model-selection tooling

Until then, this document is the explicit placeholder so the work does not get lost.