whetstone_RSA/presentations/rsa_tiny_models_talk.md

Tiny Models For Bounded NLP Decisions

whetstone_RSA
Fast-slide draft for a 5-10 minute programmer talk

This version is intentionally structured as many short slides with minimal text. The goal is to support a visual talk with one idea per slide.

Slide 1: Title

Tiny Models For Bounded NLP Decisions

whetstone_RSA

Turning messy user requests into deterministic tool calls

Visual idea: Large title, small diagram of user request -> contract -> tool

Speaker notes: This talk is about a middle layer between regex and LLMs.

Slide 2: A Common Shape

Users type messy things.

Software needs clean actions.

Visual idea: Messy speech bubble on left, clean typed API object on right

Speaker notes: This shows up all over software.

Slide 3: The Gap

We often have:

• fuzzy input
• bounded output
• deterministic execution

Visual idea: Three stacked boxes with arrows

Speaker notes: That middle combination is the important one.

Slide 4: Today’s Typical Options

• regex
• rules
• LLM prompt

Visual idea: Three boxes, with regex looking brittle and LLM looking oversized

Speaker notes: Most teams bounce between brittle and oversized.

Slide 5: Regex Is Cheap

Regex and rules are:

• fast
• local
• inspectable

Visual idea: Green checkmarks next to those three traits

Speaker notes: This is why people keep reaching for rules.

Slide 6: Regex Is Also Annoying

Regex and rules are:

• brittle
• tedious
• bad at paraphrases

Visual idea: Same rule box cracking under many user phrasings

Speaker notes: The moment users say the same thing many ways, rules get ugly.

Slide 7: LLMs Feel Great

LLMs are:

• flexible
• natural-language friendly
• good with paraphrases

Visual idea: Many different speech bubbles converging successfully

Speaker notes: This is why people like them.

Slide 8: LLMs Are Often Overkill

For bounded tasks, LLMs can be:

• expensive
• heavy
• hard to control

Visual idea: Huge engine powering a tiny gear

Speaker notes: A lot of current usage is more model than the task actually needs.

Slide 9: The Missing Middle

There should be a middle layer.

Visual idea: regex -> RSA -> LLM

Speaker notes: That middle layer is what RSA is for.

Slide 10: What RSA Is

RSA is a contract layer for bounded semantic decisions.

Visual idea: Contract sheet icon between user and tool

Speaker notes: Not a chatbot. Not a search engine. A decision contract layer.

Slide 11: What RSA Does

RSA takes:

• messy input
• bounded schema
• optional context

Visual idea: Three inputs feeding one box

Speaker notes: The schema is what keeps it narrow.

Slide 12: What RSA Returns

RSA returns:

• action
• slots
• confidence
• abstain or escalate

Visual idea: Typed JSON-like result

Speaker notes: The output is structured, not freeform prose.

Slide 13: The Core Pattern

Natural language
-> semantic interpretation
-> bounded contract
-> deterministic software

Visual idea: Simple four-step pipeline

Speaker notes: The model interprets. The software executes.

Slide 14: Example Query

show me the failed builds from yesterday

Visual idea: One big centered query bubble

Speaker notes: This is the kind of input users naturally want to type.

Slide 15: Example Contract

Becomes:

• action: search_builds
• status: failed
• date: yesterday
• confidence: 0.93

Visual idea: Search query transformed into structured object

Speaker notes: This is not generation. It is semantic interpretation into a contract.

Slide 16: Example Execution

Then ordinary software:

• runs the query
• renders the results
• logs the action

Visual idea: Database, search, and UI boxes after the contract

Speaker notes: Once the contract exists, it is just normal software again.

Slide 17: Why This Can Stay Tiny

The output is bounded.

Visual idea: Huge input cloud mapped into a small action menu

Speaker notes: The model does not need to generate arbitrary text.

Slide 18: First Real Target

whetstone_DSL

AST-first codegen pipeline

Visual idea: WhetstoneDSL box feeding RSA gates

Speaker notes: This is not a toy example. These are real pipeline decisions.

Slide 19: The Initial Gate

prereq_op_selector

Decides prerequisite operations

before a taskitem executes.

Visual idea: Taskitem -> prereq gate -> operation checklist

Speaker notes: The gate decides what must happen before execution.

Slide 20: Gate Input

Input is structured task context:

• title
• requirements
• acceptance criteria
• constraints

Visual idea: Four input cards into one gate

Speaker notes: The model sees the compact decision context, not the whole world.

Slide 21: Gate Output

Output was:

• needs_resolve_dependencies
• needs_architect_review
• needs_validate_intake

Visual idea: Three boolean switches

Speaker notes: This is the bounded contract surface.

Slide 22: Source Corpus

1000 pipeline runs

JSON artifacts

real decision logs

Visual idea: Stack of JSON files

Speaker notes: The point is extracting supervision from actual pipeline behavior.

Slide 23: Extraction

extract_gate_rows.py

turns run artifacts

into per-gate rows.

Visual idea: JSON stack -> extraction script -> row table

Speaker notes: Each row isolates one decision.

Slide 24: Clean Rows

1321 total rows

24 schema-drift rows rejected

1297 clean rows

Visual idea: Row counter with a small rejected pile

Speaker notes: Bad rows get rejected instead of quietly poisoning the gate.

Slide 25: The Surprise

needs_validate_intake

was always true.

1297 / 1297

Visual idea: One switch permanently locked on

Speaker notes: A label with zero variance is not learnable. It is a constant.

Slide 26: Gate Decomposition

The real gate became two gates:

• resolve dependencies
• architect review

Visual idea: One big gate splitting into two smaller gates

Speaker notes: The data changed the gate shape. That is the process working.

Slide 27: Tiny Specialists

One specialist per gate.

About 213K parameters.

About 800KB on disk.

Visual idea: Tiny model file next to a normal image file

Speaker notes: The size is small because the decision surface is small.

Slide 28: First Results

Baseline accuracy:

• resolve: 69.4%
• architect: 75.0%

Visual idea: Two simple bars

Speaker notes: These are not victory-lap numbers. They are diagnostic numbers.

Slide 29: Honest Diagnosis

1297 rows

but only about 17 unique text patterns.

Visual idea: Lots of rows collapsing into a few templates

Speaker notes: More rows from the same shape do not create richer signal.

Slide 30: The Fix

Better input fields:

• architecture surface
• cross-component deps
• security sensitivity
• uncertainty score

Visual idea: Taskitem schema before and after

Speaker notes: The fix is improving the contract upstream, not pretending the weak model is good.

Slide 31: Model Sizing

Match model size

to gate entropy.

Visual idea: Gate difficulty -> model tier

Speaker notes: The goal is the smallest model that clears the quality bar.

Slide 32: What Makes A Gate Hard?

• more labels
• class imbalance
• wider context
• fuzzy boundaries
• slot interactions

Visual idea: Difficulty knobs

Speaker notes: These are the factors that should predict model tier.

Slide 33: Runtime Scaffold

C++ library:

• gate definitions
• evidence
• diagnosis
• probes

Visual idea: Fabricate specialist -> RSA runtime -> WhetstoneDSL

Speaker notes: The repo is not just notes. There is a runtime scaffold.

Slide 34: Careful Claim

RSA does not replace all LLM reasoning.

RSA replaces bounded semantic decisions.

Visual idea: Bounded decisions inside scope, open-ended reasoning outside

Speaker notes: Keep the claim narrow. That makes it harder to knock down.

Slide 35: Closing

Fuzzy input in.

Typed contract out.

Deterministic software after that.

Visual idea: Final clean pipeline graphic

Speaker notes: Do not make the model own the whole task when it only needs to own the ambiguous boundary.