Initial whetstone_RSA architecture and C++ scaffold
This commit is contained in:
Bill Holcombe
391
docs/gate_failure_taxonomy.md
Normal file
391
docs/gate_failure_taxonomy.md
Normal file
@@ -0,0 +1,391 @@
|
||||
# Gate Failure Taxonomy
|
||||
|
||||
## Purpose
|
||||
|
||||
This document defines a first-pass taxonomy for why bounded decision gates fail and
|
||||
how `whetstone_RSA` should reason about those failures.
|
||||
|
||||
The goal is to make gate diagnosis a library capability rather than an ad hoc human
|
||||
judgment call.
|
||||
|
||||
This taxonomy is motivated by the `whetstone_DSL` case study, but it is written as
|
||||
an RSA-library abstraction.
|
||||
|
||||
## Core Claim
|
||||
|
||||
When a bounded gate performs poorly, the default answer should not be "use a larger
|
||||
model."
|
||||
|
||||
The library should first determine what kind of failure is happening.
|
||||
|
||||
Different failure types require different responses:
|
||||
|
||||
- some need more capacity
|
||||
- some need better confidence policy
|
||||
- some should be decomposed
|
||||
- some should become deterministic
|
||||
- some have unstable labels and should not be treated as ordinary classifiers
|
||||
|
||||
## Failure Classes
|
||||
|
||||
### 1. `capacity_limited_gate`
|
||||
|
||||
Description:
|
||||
|
||||
The gate is well-defined and reasonably stationary, but the current model tier does
|
||||
not have enough capacity to separate the decision surface reliably.
|
||||
|
||||
Characteristic symptoms:
|
||||
|
||||
- performance improves with training but plateaus below target
|
||||
- label boundaries appear coherent under manual inspection
|
||||
- confusion is broad rather than concentrated in one structural pattern
|
||||
- richer models or larger context are plausible fixes
|
||||
|
||||
Likely root causes:
|
||||
|
||||
- too little model capacity
|
||||
- too little representational depth
|
||||
- long-range dependencies exceed current context handling
|
||||
- input compression is too lossy for the chosen tier
|
||||
|
||||
Recommended probes:
|
||||
|
||||
- larger-model comparison
|
||||
- context-width probe
|
||||
- calibration probe
|
||||
- feature ablation probe
|
||||
|
||||
Recommended interventions:
|
||||
|
||||
- move to a larger specialist tier
|
||||
- add better structured features
|
||||
- use a richer encoder while preserving the same gate schema
|
||||
|
||||
### 2. `factorizable_gate`
|
||||
|
||||
Description:
|
||||
|
||||
The gate is represented as one multiclass decision, but the actual semantics are a
|
||||
composition of smaller independent or semi-independent decisions.
|
||||
|
||||
Characteristic symptoms:
|
||||
|
||||
- flat multiclass performance despite apparently simple semantics
|
||||
- classes can be expressed as Cartesian products or hierarchical branches
|
||||
- confusion concentrates between composite labels
|
||||
- downstream action is itself assembled from multiple sub-decisions
|
||||
|
||||
Likely root causes:
|
||||
|
||||
- a product decision has been collapsed into one label
|
||||
- one model is being asked to infer multiple policy axes at once
|
||||
- the current evaluation obscures easier sub-structure
|
||||
|
||||
Recommended probes:
|
||||
|
||||
- factorization probe
|
||||
- hierarchical-gate probe
|
||||
- per-axis confidence probe
|
||||
|
||||
Recommended interventions:
|
||||
|
||||
- split into multiple smaller gates
|
||||
- add a deterministic combiner over sub-gates
|
||||
- replace flat multiclass routing with hierarchical routing
|
||||
|
||||
### 3. `non_stationary_gate`
|
||||
|
||||
Description:
|
||||
|
||||
The label semantics change over time because they depend on evolving tooling,
|
||||
policies, or environment state rather than a stable ontology.
|
||||
|
||||
Characteristic symptoms:
|
||||
|
||||
- labels become stale as the surrounding system improves
|
||||
- class definitions drift without changing the input domain
|
||||
- retraining is needed because the target policy moved, not because the model forgot
|
||||
- disagreements cluster at capability boundaries rather than lexical boundaries
|
||||
|
||||
Likely root causes:
|
||||
|
||||
- output labels encode current deployment policy
|
||||
- the gate depends on tool availability or maturity
|
||||
- class meaning is roadmap-dependent
|
||||
|
||||
Recommended probes:
|
||||
|
||||
- schema-stability probe
|
||||
- temporal re-label probe
|
||||
- policy-variant probe
|
||||
|
||||
Recommended interventions:
|
||||
|
||||
- separate stable ontology fields from changing policy fields
|
||||
- turn part of the decision into a policy layer above the model
|
||||
- use hierarchical routing where the first stage predicts a stable property and a
|
||||
later stage applies current deployment policy
|
||||
|
||||
### 4. `missing_context_gate`
|
||||
|
||||
Description:
|
||||
|
||||
The gate might be learnable, but the current input encoding omits the context needed
|
||||
to make the decision safely.
|
||||
|
||||
Characteristic symptoms:
|
||||
|
||||
- short-text performance is poor but humans request surrounding context
|
||||
- errors cluster on inputs that depend on hidden project or workflow state
|
||||
- adding metadata is expected to help more than scaling parameters
|
||||
- the same surface text would require different answers in different contexts
|
||||
|
||||
Likely root causes:
|
||||
|
||||
- important state is absent from the input
|
||||
- text summaries collapse policy-critical structure
|
||||
- the gate implicitly depends on project metadata, not just user language
|
||||
|
||||
Recommended probes:
|
||||
|
||||
- context-width probe
|
||||
- metadata augmentation probe
|
||||
- stateful-context probe
|
||||
|
||||
Recommended interventions:
|
||||
|
||||
- enrich inputs with structured metadata
|
||||
- separate text understanding from stateful decision logic
|
||||
- require abstention when needed context is unavailable
|
||||
|
||||
### 5. `deterministic_disguised_as_ml_gate`
|
||||
|
||||
Description:
|
||||
|
||||
The gate looks statistical at first glance, but the output is largely determined by
|
||||
known rules or a stable formula.
|
||||
|
||||
Characteristic symptoms:
|
||||
|
||||
- very high performance with synthetic or structural encodings
|
||||
- human explanation reduces to a crisp rule
|
||||
- model errors mostly reflect encoding mismatch, not true ambiguity
|
||||
- downstream users already trust a formula more than the classifier
|
||||
|
||||
Likely root causes:
|
||||
|
||||
- a deterministic rule was not implemented yet
|
||||
- the learned model is being used as a proxy for explicit policy
|
||||
- training data was easier to generate than writing the rule
|
||||
|
||||
Recommended probes:
|
||||
|
||||
- deterministic-rule probe
|
||||
- symbolic baseline probe
|
||||
- feature-rule extraction probe
|
||||
|
||||
Recommended interventions:
|
||||
|
||||
- replace with deterministic logic
|
||||
- keep the model only as a convenience wrapper over explicit rules
|
||||
- use the learned gate only when the rule inputs are partially missing
|
||||
|
||||
### 6. `label_space_mismatch`
|
||||
|
||||
Description:
|
||||
|
||||
The gate's labels are underspecified, overlapping, misordered, or otherwise not
|
||||
aligned with the actual decision boundary the system needs.
|
||||
|
||||
Characteristic symptoms:
|
||||
|
||||
- confusion clusters around neighboring or semantically overlapping labels
|
||||
- multiple labels seem equally valid for the same input
|
||||
- label guidelines are hard to explain concisely
|
||||
- accuracy does not reflect practical usefulness because some "mistakes" are close
|
||||
|
||||
Likely root causes:
|
||||
|
||||
- labels mix ontology and policy
|
||||
- labels are too coarse or too fine
|
||||
- adjacent classes are not operationally distinct
|
||||
|
||||
Recommended probes:
|
||||
|
||||
- confusion-structure probe
|
||||
- class-merge probe
|
||||
- hierarchical-label probe
|
||||
|
||||
Recommended interventions:
|
||||
|
||||
- merge or split labels
|
||||
- add hierarchy to the label space
|
||||
- redefine evaluation so high-cost confusions are distinguished from near-miss ones
|
||||
|
||||
### 7. `guardrail_limited_gate`
|
||||
|
||||
Description:
|
||||
|
||||
The gate is good enough to be useful, but not good enough to be trusted without a
|
||||
deployment policy around it.
|
||||
|
||||
Characteristic symptoms:
|
||||
|
||||
- raw accuracy is moderate but confidence is informative
|
||||
- abstention can isolate hard cases
|
||||
- downstream checks can catch many residual errors
|
||||
- accepted predictions are much stronger than overall predictions
|
||||
|
||||
Likely root causes:
|
||||
|
||||
- the model is solving an economically useful subset of the problem
|
||||
- deployment policy has not yet been calibrated
|
||||
|
||||
Recommended probes:
|
||||
|
||||
- confidence-threshold probe
|
||||
- abstain-coverage probe
|
||||
- retry-recovery probe
|
||||
- deterministic-check catch-rate probe
|
||||
|
||||
Recommended interventions:
|
||||
|
||||
- deploy with guardrails
|
||||
- set confidence thresholds by risk tier
|
||||
- add retry and escalation paths
|
||||
|
||||
## Diagnostic Probes
|
||||
|
||||
The taxonomy is only useful if the library has standard probes.
|
||||
|
||||
### `confidence_threshold_probe`
|
||||
|
||||
Question:
|
||||
|
||||
Does the gate become safe and useful when low-confidence cases abstain?
|
||||
|
||||
Outputs:
|
||||
|
||||
- accept rate
|
||||
- accuracy on accepted
|
||||
- silent error estimate
|
||||
|
||||
### `factorization_probe`
|
||||
|
||||
Question:
|
||||
|
||||
Can a weak multiclass gate be re-expressed as multiple smaller gates with a better
|
||||
combined policy?
|
||||
|
||||
Outputs:
|
||||
|
||||
- candidate factorization structure
|
||||
- estimated per-axis difficulty
|
||||
- combined accuracy or accepted-precision comparison
|
||||
|
||||
### `deterministic_rule_probe`
|
||||
|
||||
Question:
|
||||
|
||||
Is the gate mostly encoding a crisp rule that should live in ordinary software?
|
||||
|
||||
Outputs:
|
||||
|
||||
- symbolic baseline score
|
||||
- overlap between model errors and rule violations
|
||||
|
||||
### `schema_stability_probe`
|
||||
|
||||
Question:
|
||||
|
||||
Are the labels stable over time, or do they move with deployment policy and tool
|
||||
availability?
|
||||
|
||||
Outputs:
|
||||
|
||||
- stability rating
|
||||
- ontology-vs-policy split recommendation
|
||||
|
||||
### `context_width_probe`
|
||||
|
||||
Question:
|
||||
|
||||
Is performance limited by missing project or workflow context?
|
||||
|
||||
Outputs:
|
||||
|
||||
- delta from richer context
|
||||
- recommended minimum context set
|
||||
|
||||
### `confusion_structure_probe`
|
||||
|
||||
Question:
|
||||
|
||||
Are the observed errors random, adjacent, hierarchical, or compositional?
|
||||
|
||||
Outputs:
|
||||
|
||||
- confusion topology
|
||||
- merge, split, or hierarchy recommendation
|
||||
|
||||
## Recommended Response Order
|
||||
|
||||
The library should approach gate triage in this order:
|
||||
|
||||
1. Check whether the gate is deterministic in disguise.
|
||||
2. Check whether the label space is stable and coherent.
|
||||
3. Check whether the gate is factorizable.
|
||||
4. Check whether critical context is missing.
|
||||
5. Check whether a guarded deployment policy is already sufficient.
|
||||
6. Only then treat scale as the primary lever.
|
||||
|
||||
## Library Implications
|
||||
|
||||
To support this taxonomy, `whetstone_RSA` should eventually represent each gate with
|
||||
enough metadata for diagnosis.
|
||||
|
||||
Suggested gate metadata:
|
||||
|
||||
- `gate_id`
|
||||
- `label_schema`
|
||||
- `label_stability`
|
||||
- `task_family`
|
||||
- `input_modalities`
|
||||
- `supports_abstain`
|
||||
- `deterministic_baseline_available`
|
||||
- `candidate_factorizations`
|
||||
- `risk_tier`
|
||||
|
||||
Suggested diagnostic result shape:
|
||||
|
||||
```text
|
||||
GateDiagnosis
|
||||
primary_failure_class
|
||||
secondary_failure_classes
|
||||
supporting_signals
|
||||
recommended_probes
|
||||
recommended_interventions
|
||||
confidence
|
||||
```
|
||||
|
||||
## Case-Study Motivators
|
||||
|
||||
The case study motivates this taxonomy, but does not constrain it.
|
||||
|
||||
- A gate like `prereq_op` suggests `factorizable_gate`
|
||||
- A gate like `automatability` suggests `non_stationary_gate`
|
||||
- A gate like `confidence_tier` suggests `deterministic_disguised_as_ml_gate`
|
||||
- A gate like `verification_type` or `worker_type` may become
|
||||
`guardrail_limited_gate`
|
||||
|
||||
These examples are evidence of recurring patterns, not bespoke exceptions.
|
||||
|
||||
## Summary
|
||||
|
||||
The main architectural insight is simple:
|
||||
|
||||
Bounded-gate failure is not one problem.
|
||||
|
||||
If the RSA library can classify failure types before scaling models blindly, it will
|
||||
be cheaper, safer, and easier to generalize across domains.
|
||||
Reference in New Issue
Block a user