diff --git a/docs/diagnostic_patterns.md b/docs/diagnostic_patterns.md new file mode 100644 index 0000000..4537796 --- /dev/null +++ b/docs/diagnostic_patterns.md @@ -0,0 +1,434 @@ +# RSA Diagnostic Patterns + +## Purpose + +This document is a named-pattern library for diagnosing bounded decision gates. + +Each pattern has: +- a name that can be referenced in conversations and code +- a concrete observable signal +- the mechanism that produces that signal +- a detection test +- a specific fix +- the conditions under which the fix fails +- a real example from the `whetstone_DSL` case study where one exists + +This is not the same as the failure taxonomy (`gate_failure_taxonomy.md`). +The taxonomy classifies failure _types_. This document gives you something to +actually do when you observe a specific symptom. + +**Intended audience:** developers, model-assisted debuggers, automated diagnosis +pipelines. Each pattern is written at a level of specificity that a smaller model +or a programmer without deep ML background can apply correctly if they can +recognize the signal. + +--- + +## Pattern Index + +| Name | Signal | Fix | +|------|--------|-----| +| [Directional Collapse](#1-directional-collapse) | One class maps >80% to one other specific class | Merge those classes or route them to a prior stage | +| [Constant Class](#2-constant-class) | One label fires on 100% (or near-100%) of corpus rows | Remove label from gate; encode as a rule | +| [Corpus Uniformity Trap](#3-corpus-uniformity-trap) | High row count but very few unique input texts | Discard corpus; find a richer data source | +| [Capacity Plateau](#4-capacity-plateau) | Larger model tier does not improve over smaller | Stop scaling; diagnose label or data quality | +| [Schema Drift Contamination](#5-schema-drift-contamination) | Structured field contains out-of-distribution values from another class | Hard-filter before training | +| [Upstream Feature Starvation](#6-upstream-feature-starvation) | Gate performance doesn't improve with any intervention | Check whether discriminative fields exist at inference time | +| [Axis Factorization](#7-axis-factorization) | Multi-class gate is weak; classes can be expressed as a Cartesian product | Decompose into binary sub-gates with a deterministic combiner | +| [Grokking Phase Transition](#8-grokking-phase-transition) | Accuracy is flat for thousands of steps, then jumps sharply | Do not stop training at the plateau | +| [Guardrail Rescue](#9-guardrail-rescue) | Raw accuracy is moderate but confidence is informative | Deploy with an abstain threshold rather than scaling | +| [Label Boundary Drift](#10-label-boundary-drift) | Confusion is broad and inconsistent; manual review disagrees on labeling | Redefine labels, not model | + +--- + +## 1. Directional Collapse + +**Signal:** In the confusion matrix, one class maps overwhelmingly (>80%) to one +other specific class. The confusion is directional and concentrated, not diffuse. + +**Mechanism:** The two classes are indistinguishable in the input representation. +The model has learned everything it can and is guessing the majority direction. +Scaling or adding data will not help unless you change what features the gate sees. + +This is different from random confusion (which is diffuse) and from capacity +limitation (which produces broad, scattered errors). + +**Detection test:** Look at the per-class confusion. If class A sends >80% of its +errors to exactly one destination class B, and this persists across model tiers, +you have Directional Collapse, not a capacity problem. + +**Fix:** Merge A and B into a single bucket. If the downstream system needs to +distinguish them, add a prior routing stage that can separate them from the rest +of the label space — then apply a second gate only to that bucket. The two-stage +decomposition is the standard fix. + +**When the fix fails:** If the merged bucket is large and the downstream system +genuinely needs the A/B distinction, you have a feature enrichment problem, not a +model problem. The fix is to add a feature to the input schema that carries the +A/B signal, then retrain. + +**Taxonomy link:** `label_space_mismatch` in `gate_failure_taxonomy.md`. + +**Example — `automatability` gate, April 2026:** +The `deterministic` class sent 100% of its errors to `template` in the medium-tier +model. Adding data and scaling both failed. The fix was a two-stage decomposition: +stage 1 (binary) separates `{deterministic, template}` from `{specialist, slm, llm, +human}`; stage 2 (4-way) classifies within the routed set. Both stages achieved 100% +accuracy. The `deterministic`/`template` distinction turns out to require information +not present in taskitem titles — that distinction belongs in a richer input schema, +not in the current gate. + +--- + +## 2. Constant Class + +**Signal:** One label in your label schema fires on 100% (or near-100%) of all rows +in the corpus. It is always true regardless of input. + +**Mechanism:** The label is not a decision surface — it is a structural property of +the corpus or the system. Every example in this data source satisfies the condition, +so no model can learn when it _doesn't_ fire. + +**Detection test:** Compute per-label frequency over the full corpus. If any label +is present in ≥95% of rows, apply this pattern. + +**Fix:** Remove the label from the gate's label space. If the condition is always +true in the current system, encode it as a rule (`always apply X`). If it may +eventually become false as the system evolves, defer the gate until real negative +examples exist. + +**When the fix fails:** If the label was constant only in the current corpus and +will have genuine variance in a future corpus (different data sources, different +system states), removing it may be premature. Verify by asking whether a different +project or data source would produce negative examples. + +**Taxonomy link:** `deterministic_disguised_as_ml_gate` in `gate_failure_taxonomy.md`. + +**Example — `needs_validate_intake` field, April 2026:** +Every one of 1,297 accepted rows had `needs_validate_intake = True`. The field was +in the label space because it appeared to be a decision output, but inspection +showed it was always True in the corpus. It was dropped from the label space. The +effective gate became two binary classifiers over the remaining fields. + +--- + +## 3. Corpus Uniformity Trap + +**Signal:** The corpus has high row count but very few unique input texts. For +example: 1,297 rows with only 17 unique texts. + +**Mechanism:** The data source generates many rows from a small number of canonical +templates. The apparent training set size is misleading — the model is effectively +seeing a handful of inputs repeated. Variance in outputs cannot be attributed to +variance in inputs, so the model cannot learn a discriminative function. + +**Detection test:** Count unique input texts. If `unique_texts / total_rows < 0.1`, +or if unique text count is below ~50 regardless of total rows, you are in this trap. + +**Fix:** Find a different data source that has genuine input variance. Do not +generate more rows from the same corpus. Do not oversample or augment the existing +rows. The problem is the source, not the quantity. + +**When the fix fails:** If the only available data is template-driven (e.g., all +examples come from the same intake spec format), this pattern is a blocking +constraint on the gate. The gate cannot be trained until the input schema or data +source provides discriminative variance. Document this as a `missing_context_gate` +and treat data collection as a prerequisite milestone. + +**Taxonomy link:** `missing_context_gate` in `gate_failure_taxonomy.md`. + +**Example — `prereq_op` gate, April 2026:** +The whetstone_DSL run corpus produced 1,297 rows but only 17 unique context texts. +All tasks generated from the same intake spec had identical context text. The +discriminative signal (step-level description) was never stored in the run artifact. +Resolution: add RSA-relevant fields to the taskitem schema so that each task is +self-contained. Collection of real training data is blocked until that schema change. + +--- + +## 4. Capacity Plateau + +**Signal:** A larger model tier (e.g., medium vs. small_plus) does not improve +accuracy over the smaller tier, or slightly regresses. The performance gap between +tiers is within noise. + +**Mechanism:** The bottleneck is not model capacity — it is data quality, label +noise, or feature representation. More parameters cannot overcome bad labels or +indistinguishable input features. The model has already saturated what the data +supports. + +**Detection test:** If tier comparison shows improvement < 2pp from small_plus to +medium, and confusion patterns are similar across tiers, apply this pattern. + +**Fix:** Stop scaling. Diagnose label quality, input feature richness, and corpus +diversity instead. In priority order: +1. Check for Directional Collapse (Pattern 1) +2. Check for Corpus Uniformity Trap (Pattern 3) +3. Check for Upstream Feature Starvation (Pattern 6) +4. Check for Axis Factorization (Pattern 7) + +**When the fix fails:** If none of the above patterns apply and the gate genuinely +requires more representational power, scale is the right lever. But this should be +the last resort, not the first response. + +**Taxonomy link:** Capacity plateau is a meta-signal that rules out +`capacity_limited_gate` and shifts suspicion to other failure classes. + +**Example — `worker_type` and `automatability` gates, April 2026:** +Both gates showed near-zero improvement (and slight regression) from small_plus to +medium. For `automatability`, the root cause was Directional Collapse. For +`worker_type`, the root cause is label noise from heuristic classification — the +heuristic assigns ambiguous titles based on keyword matching, producing inconsistent +boundaries at training time. + +--- + +## 5. Schema Drift Contamination + +**Signal:** A structured output field contains values that belong to a different +schema or a different version of the contract. For example, a field that should +contain `{standard, needs_review, needs_approval, full_gates}` instead contains +`whetstone_generate_taskitems` or `manual-approval` in a subset of rows. + +**Mechanism:** The upstream system that populated the field was in a different mode +(fallback behavior, older version, or a tool that doesn't respect the current +contract). The contamination rows are not wrong in the training data sense — they +are accurately labeled artifacts of a real system state — but they represent a +different underlying distribution. + +**Detection test:** For each structured field, enumerate all unique values and check +for out-of-vocabulary entries against the current label schema. + +**Fix:** Hard-filter contaminated rows before training. Do not model drift as a +separate class unless you are deliberately building a drift detector. If you want +to handle drift gracefully at inference time, build a separate pre-filter gate. + +**When the fix fails:** If the fraction of contaminated rows is large (>20%), the +root cause is likely a systemic upstream problem. Filtering removes the symptom but +not the cause. Document the contamination source and fix the upstream system. + +**Taxonomy link:** `non_stationary_gate` in `gate_failure_taxonomy.md`. + +**Example — `prereq_op` extraction, April 2026:** +Two contamination patterns were found in the extracted corpus: tool-ID drift +(12 rows with `whetstone_generate_taskitems` etc. in `prerequisiteOps`) and +`manual-approval` drift (12 rows with an undocumented op name). Both were +hard-filtered. The extraction tool correctly flagged these via rejection logic. + +--- + +## 6. Upstream Feature Starvation + +**Signal:** Gate accuracy does not improve regardless of model tier, data volume, +or label quality interventions. The gate looks solvable in principle but resists +all fixes. + +**Mechanism:** The information needed to make the decision correctly does not exist +in the input schema at inference time. The model is not failing to learn — it is +being asked to infer a signal that was never provided. This is not a model problem. + +**Detection test:** Ask: "If a human were shown only what the model sees at +inference time, could the human reliably make this decision?" If the answer is no, +this pattern applies. Also: check whether discriminative fields are present in +training data sources but absent from the inference-time schema. + +**Fix:** Enrich the upstream schema so the discriminative fields are available when +the gate runs. This is a schema change, not a model change. Retraining after the +schema change will likely produce rapid improvement. + +**When the fix fails:** If the upstream system cannot practically carry the required +fields (e.g., they depend on information only available later in the pipeline), the +gate may need to move downstream to where the information exists, or abstain and +escalate until context is available. + +**Taxonomy link:** `missing_context_gate` in `gate_failure_taxonomy.md`. + +**Example — `prereq_op` and `automatability` gates, April 2026:** +The RSA analysis found that the taskitem schema does not carry fields like +`architectural_surface`, `cross_component_deps`, or `security_sensitive` at +inference time. These are the discriminative features for several gates. +`generate_taskitems` makes 10+ separable decisions in one LLM call; the RSA +layer cannot see the intermediate reasoning. Conclusion: the taskitem schema must +carry RSA-relevant fields as first-class outputs of `generate_taskitems` before +reliable training data can be collected for most gates. + +--- + +## 7. Axis Factorization + +**Signal:** A multi-class gate has weak accuracy despite apparently simple semantics. +The classes can be described as a Cartesian product or hierarchical branching of two +or more independent binary conditions. + +**Mechanism:** The gate is being asked to make multiple decisions at once with a +single output label. The flat label space hides compositional structure. A model +cannot reliably learn a joint decision that is actually two independent decisions +mixed together. + +**Detection test:** Try to describe each class as a combination of simpler +conditions. If you can express the full label set as `condition_A × condition_B`, +factorization applies. Also look for confusion that concentrates between classes +that share one axis but differ on the other. + +**Fix:** Decompose into one binary gate per axis. Add a deterministic combiner that +maps the binary outputs back to the original multi-class label. Evaluate the +combined accuracy against the flat gate. + +**When the fix fails:** If the axes are not independent (the output of one +sub-decision changes the valid space of the other), full factorization is not safe. +In that case, use hierarchical routing instead: a first gate routes to a subset, +a second gate classifies within that subset. + +**Taxonomy link:** `factorizable_gate` in `gate_failure_taxonomy.md`. + +**Example — `prereq_op` gate, April 2026:** +The four classes (`standard`, `needs_review`, `needs_approval`, `full_gates`) are +the Cartesian product of two binary axes: `requires_architect_review` and +`requires_manual_approval`. Decomposing into two binary classifiers and combining +deterministically gives combined accuracy directly comparable to the flat gate, +with the added benefit that each sub-gate is independently trainable, debuggable, +and deployable. + +--- + +## 8. Grokking Phase Transition + +**Signal:** Accuracy is flat or near-flat for a long training run, then jumps +sharply by 10–20pp in a short window. The jump is sustained — the model does not +regress after the transition. + +**Mechanism:** The model is slowly building internal representations before they +cross a threshold of usefulness. This is a known phenomenon in small transformers +trained on structured tasks. The plateau is not stagnation — it is slow +generalization development. Stopping during the plateau will miss the transition. + +**Detection test:** If accuracy is stagnant for thousands of steps but loss is +still decreasing, you may be approaching a transition. A sudden jump in accuracy +with sustained improvement confirms this pattern. + +**Fix:** Do not stop training at the plateau. Set a patience window or use loss +as the primary stopping criterion rather than accuracy. Run long enough to confirm +whether a transition occurs. + +**When the fix fails:** If loss stops decreasing and accuracy remains flat, the +plateau is genuine convergence on a suboptimal local minimum — not a pre-transition +state. In that case, grokking is not occurring and you should investigate data or +architecture. + +**Taxonomy link:** Not a failure pattern — a learning dynamic. May initially look +like `capacity_limited_gate` but is resolved by training duration, not model size. + +--- + +## 9. Guardrail Rescue + +**Signal:** Raw accuracy is moderate (60–80%) but threshold analysis shows that +accuracy on high-confidence predictions is substantially higher (>90%), with +acceptable abstain rates at the required threshold. + +**Mechanism:** The gate is solving a tractable subset of the problem confidently +and correctly. The remaining cases are genuinely hard — not model failures, but +irreducible uncertainty from the input. A deployment policy that abstains on low +confidence provides real value even without improving the model. + +**Detection test:** Check threshold analysis: `accuracy_on_accepted` at `threshold=0.9` +vs. `raw_accuracy`. If the gap is >10pp with `abstain_rate < 0.30`, Guardrail Rescue +applies. The gate is usable today. + +**Fix:** Deploy with an abstain threshold calibrated to the gate's risk tier. Route +abstentions to a retry (with enriched context) or escalation path. Do not block +deployment on achieving target raw accuracy if the guardrail profile is acceptable. + +**When the fix fails:** If high-confidence predictions are not substantially better +than raw accuracy (confidence is not calibrated or informative), the model is +overconfident and guardrail deployment is not safe. Check calibration before +deploying with guardrails. + +**Taxonomy link:** `guardrail_limited_gate` in `gate_failure_taxonomy.md`. + +**Example — `automatability` small_plus, April 2026:** +Raw accuracy was 69.6%, but at threshold=0.9, accuracy on accepted predictions +was 100% — with 30.4% abstain rate. The gate correctly handles 70% of inputs +with perfect precision. The abstentions are the genuinely hard cases (primarily +`deterministic` and `template` boundary). This profile was superseded by the +two-stage decomposition, but the guardrail analysis would have been valid as an +intermediate deployment. + +--- + +## 10. Label Boundary Drift + +**Signal:** Confusion is broad and scattered — no single directional pattern +dominates. Manual review of errors produces disagreement between reviewers about +the correct label. Retraining on more data does not converge. + +**Mechanism:** The label definitions are ambiguous or inconsistent. The model is +learning to imitate noisy, inconsistent labels. It is performing correctly against +the training signal it has — the training signal is the problem. + +**Detection test:** Manually review 20–30 errors. If reviewers disagree on the +correct label for more than 30% of examples, Label Boundary Drift applies. +Also: if heuristic-labeled data and human-labeled data disagree in systematic ways, +this is the pattern. + +**Fix:** Rewrite the label definitions. Add concrete decision criteria that resolve +ambiguous cases. Relabel a sample and check inter-annotator agreement before +retraining. This is a labeling problem, not a model problem. + +**When the fix fails:** If the boundary is genuinely contextual (the same input +requires different labels in different system states), the labels may need to +incorporate system state as a feature. See also Upstream Feature Starvation +(Pattern 6). + +**Taxonomy link:** `label_space_mismatch` in `gate_failure_taxonomy.md`. + +**Example — `worker_type` gate, ongoing:** +The gate uses heuristic keyword classification to generate labels. Tasks like +"Snapshot persistence (sidecar integration)" or "Compliance policy binding engine" +are labeled `implementer` by the heuristic but the model predicts `architect` or +`reviewer`. Both predictions are arguable from the title alone. The heuristic +assigns based on keyword absence (implementer is the default), which produces +unreliable labels at the boundary. Resolution requires either human labeling of +ambiguous cases or richer input features that remove the ambiguity. + +--- + +## How To Use This Document + +When a gate is underperforming, check the patterns in this order: + +1. **Look at the confusion matrix.** Is any class sending >80% of errors to one + specific other class? → [Directional Collapse](#1-directional-collapse) + +2. **Look at label frequency.** Is any label firing on >95% of corpus rows? + → [Constant Class](#2-constant-class) + +3. **Count unique input texts.** Is `unique_texts / total_rows < 0.1`? + → [Corpus Uniformity Trap](#3-corpus-uniformity-trap) + +4. **Compare tiers.** Did the larger model fail to improve? + → [Capacity Plateau](#4-capacity-plateau), then diagnose further + +5. **Check the input schema.** Does the information needed for the decision + actually exist in the input at inference time? + → [Upstream Feature Starvation](#6-upstream-feature-starvation) + +6. **Check structured fields.** Do any fields contain out-of-vocabulary values? + → [Schema Drift Contamination](#5-schema-drift-contamination) + +7. **Try to describe the classes as a Cartesian product.** + → [Axis Factorization](#7-axis-factorization) + +8. **Check threshold analysis.** Does high-confidence accuracy substantially + exceed raw accuracy? + → [Guardrail Rescue](#9-guardrail-rescue) + +9. **Manually review errors.** Do reviewers disagree on the labels? + → [Label Boundary Drift](#10-label-boundary-drift) + +10. **Check training curves.** Was the model stopped during a long plateau before + a possible grokking transition? + → [Grokking Phase Transition](#8-grokking-phase-transition) + +Scale (larger model, more data) is not a pattern. It is the default assumption +that these patterns exist to refute.