102 lines
3.2 KiB
Markdown
102 lines
3.2 KiB
Markdown
# whetstone_RSA
|
|
|
|
`whetstone_RSA` is a library-first project for bounded natural-language-to-decision
|
|
systems.
|
|
|
|
The core premise is simple:
|
|
|
|
- use learned routing only at the ambiguity boundary
|
|
- keep the output space explicitly enumerable
|
|
- hand execution to deterministic software as early as possible
|
|
|
|
This project grows out of two existing lines of work:
|
|
|
|
- `WhetstoneAI_Fabricate`: routed-specialist architecture and tiny transformer experiments
|
|
- `CLionProjects/whetstone_DSL`: AST-first IDE and MCP pipeline with bounded decisions
|
|
|
|
## Problem Statement
|
|
|
|
LLMs are often being used for tasks that are not truly open-ended:
|
|
|
|
- choose one workflow from a small set
|
|
- fill a known form schema
|
|
- classify a request into a bounded enum
|
|
- pick an AST action from a valid operation set
|
|
- route a request to a deterministic tool
|
|
|
|
Those tasks do not need internet-scale latent knowledge. They need:
|
|
|
|
- a bounded decision vocabulary
|
|
- lightweight contextual inference
|
|
- validation and abstention
|
|
- deterministic downstream execution
|
|
|
|
`whetstone_RSA` exists to turn that into a reusable runtime rather than a one-off
|
|
experiment.
|
|
|
|
## Current Hypothesis
|
|
|
|
There is a measurable relationship between task entropy and the smallest model that
|
|
can fill a decision gate reliably.
|
|
|
|
The practical goal is not "make the model bigger until it works." The goal is:
|
|
|
|
- estimate gate entropy
|
|
- choose the smallest model family that clears the quality target
|
|
- escalate to a larger model only when the entropy budget requires it
|
|
|
|
This should let downstream applications keep model scope aligned with the actual
|
|
decision surface they are filling.
|
|
|
|
## Initial Scope
|
|
|
|
The first target is a reusable library that supports three modes:
|
|
|
|
- `classify`: pick one label from a fixed enum
|
|
- `fill`: populate a bounded structured schema
|
|
- `route`: choose a deterministic tool/template/action and return typed arguments
|
|
|
|
Each run should produce:
|
|
|
|
- selected action or label
|
|
- filled slots
|
|
- confidence
|
|
- abstain or escalate decision
|
|
- trace metadata suitable for evaluation
|
|
|
|
## First Case Study
|
|
|
|
The first case study is the WhetstoneDSL specialist fleet in:
|
|
|
|
- `/home/bill/Documents/CLionProjects/whetstone_DSL/specialists`
|
|
|
|
That work already established:
|
|
|
|
- ~800KB transformer specialists are viable for bounded editor decisions
|
|
- some tasks fit comfortably in that footprint
|
|
- other gates appear too entropic for the smallest tier and need larger variants
|
|
|
|
This project turns that observation into a formal sizing framework instead of
|
|
handling it ad hoc.
|
|
|
|
## Project Layout
|
|
|
|
- `docs/architecture.md`: library boundaries and runtime model
|
|
- `docs/case_studies/whetstone_dsl.md`: current grounding case study
|
|
- `docs/entropy_model_sizing.md`: baseline research plan for entropy-to-model sizing
|
|
|
|
## Near-Term Deliverables
|
|
|
|
1. Define the core decision schema and runtime API.
|
|
2. Inventory WhetstoneDSL gates by output vocabulary, ambiguity, and failure mode.
|
|
3. Build a baseline entropy score for each gate.
|
|
4. Train and compare specialist sizes against those gates.
|
|
5. Turn the result into a library other local projects can call.
|
|
|
|
## Design Rule
|
|
|
|
Learned behavior should live only where deterministic tooling is not yet enough.
|
|
|
|
Everything after a bounded decision should be deterministic unless there is a clear
|
|
reason not to do that.
|