diff --git a/presentations/whetstoneRSA_full_story_deck.html b/presentations/whetstoneRSA_full_story_deck.html new file mode 100644 index 0000000..9060160 --- /dev/null +++ b/presentations/whetstoneRSA_full_story_deck.html @@ -0,0 +1,935 @@ + + + + + +WhetstoneRSA - When LLMs Are Overkill + + + +
+ +
+
WhetstoneRSA
+
+

Product Spec In. Program Out.

+

Then make the planning layer local, cheaper, and repeatable.

+
+
1main claim
+
+ +
+
The thing that matters
+
+

A working program in about five minutes.

+

That is already a different software-building experience.

+
+
2speed is the hook
+
+ +
+
Comparison I watched
+
+
+
30minutes agent loop
+
5minutes AST-first pipeline
+
+

Same general class of app-building demo.

+
+
3rough observed scale
+
+ +
+
Important distinction
+
+

AST-first generation created the speedup.

+

RSA attacks the remaining repeated LLM decisions.

+
+
4keep the claim clean
+
+ +
+
Why AST-first is fast
+
+
    +
  • plan into structured work
    • +
  • generate from AST shape
    • +
  • validate against contracts
    • +
  • avoid long wandering agent loops
    • +
+
+
5structure beats wandering
+
+ +
+
The remaining model work
+
+

The pipeline still has planning decisions.

+

Task splits, target files, required tools, acceptance checks, prep gates.

+
+
6where RSA fits
+
+ +
+
The starting point
+
+

LLMs are amazing.

+

But many software decisions are smaller than an LLM.

+
+
7not anti-LLM
+
+ +
+
The shape
+
+
The input can be huge.
+
The output is small.
+
+
3the key distinction
+
+ +
+
Example request
+
+

show me the failed builds from yesterday

+
+
4messy human phrasing
+
+ +
+
Same intent, many phrasings
+
+
    +
  • what broke in CI yesterday
    • +
  • pull up yesterday's red builds
    • +
  • did the pipeline fail last night
    • +
  • which build jobs failed since yesterday
    • +
+
+
5input variety
+
+ +
+
What the app actually needs
+
+
+
action
search_builds
+
status
failed
+
date
yesterday
+
+
+
6small structured output
+
+ +
+
Output scale
+
+ 
5 actions x 4 statuses x 4 date buckets
+= about 80 structured outputs
+
+
7small menu
+
+ +
+
Classical code starts well
+
+ 
if text has "failed" and "build":
+    action = search_builds
+
+if text has "yesterday":
+    date = yesterday
+
+
8simple at first
+
+ +
+
Then regex pays the human tax
+
+

You enumerate the input side.

+

Every new phrase becomes another pattern, exception, or bug.

+
+
9input enumeration
+
+ +
+
Rule explosion
+
+ 
5 actions
+x 10 common phrasings
+x 4 statuses
+x 8 time phrasings
+= 1,600-ish rule cases
+
+
10conservative example
+
+ +
+
Regex tradeoff
+
+
    +
  • excellent output control
    • +
  • tiny runtime cost
    • +
  • input coverage gets expensive
    • +
+
+
11great gate, painful input
+
+ +
+
LLMs feel easy
+
+

They understand the messy input.

+

One prompt can replace a pile of brittle phrase rules.

+
+
12the appeal is real
+
+ +
+
But the scale is wrong
+
+

A giant general model

+

choosing from a tiny application menu.

+
+
13overkill
+
+ +
+
API runtime scale
+
+ + + + + +
ModelInputOutput
gpt-5.4-mini$0.75 / 1M tokens$4.50 / 1M tokens
gpt-5.4$2.50 / 1M tokens$15 / 1M tokens
gpt-5.5$5 / 1M tokens$30 / 1M tokens
+
+
14OpenAI pricing, checked Apr 2026
+
+ +
+
WhetstoneDSL example set
+
+
+
100easy programs
+
25challenging programs
+
+

125 product specs used as generation/evaluation examples.

+
+
15the benchmark scale
+
+ +
+
Measured extraction slice
+
+ + + + + + +
ArtifactCount
runs999
taskitems1,321
extracted gate rows5,284
gates per taskitem4 extracted
+
+
16local extracted data
+
+ +
+
125 programs implies
+
+ 
125 programs x 1.32 taskitems/program
+= ~165 taskitems
+
+~165 taskitems x 10+ planning decisions
+= ~1,650+ decisions
+
+
17planning decision count
+
+ +
+
Planning-only token estimate
+
+ 
125 specs:
+~56K planning input tokens
+~99K planning output tokens
+
+This is before implementation calls.
+
+
18planning layer only
+
+ +
+
125-program planning cost
+
+ + + + + +
ModelPlanning-only estimate
gpt-5.4-mini~$0.49
gpt-5.4~$1.63
gpt-5.5~$3.25
+
+
19small demo, real meter
+
+ +
+
Add implementation calls
+
+ 
165 taskitems
+x 5K input + 2K output tokens
+= ~826K input + ~330K output
+
+Still a modest example.
+
+
20implementation dominates
+
+ +
+
Estimated API cost
+
+ + + + + +
ModelModerate tasksHeavier tasks
gpt-5.4-mini~$2.60~$6.70
gpt-5.4~$8.65~$22.30
gpt-5.5~$17.30~$44.60
+
+
21one 125-program batch
+
+ +
+
Scale changes the economics
+
+

At codebase scale, token spend becomes infrastructure spend.

+

Retries, larger context, regressions, and many users multiply the meter.

+
+
22the bill compounds
+
+ +
+
Team-scale intuition
+
+ 
$20 per substantial generation run
+x 10,000 runs/month
+= $200,000/month
+
+before engineer review time
+
+
23illustrative scale
+
+ +
+
Different cost shape
+
+

From recurring token meter

+

toward a few good GPUs, local gates, and occasional subscriptions.

+
+
23ownership vs meter
+
+ +
+
My original plan
+
+

Use LoRA on a local SLM.

+

Adapt a small language model to WhetstoneDSL decisions.

+
+
16the first instinct
+
+ +
+
LoRA still carries the base model
+
+
    +
  • 1B / 3B / 7B parameter base
    • +
  • adapter is small
    • +
  • runtime model is still big
    • +
+
+
17adapter small, base still large
+
+ +
+
The better question
+
+

Was the task ever large enough to deserve that model?

+
+
18question the premise
+
+ +
+
Where this came from
+
+

WhetstoneDSL

+

AST-first programming and universal transpiling

+
+
19creation story
+
+ +
+
What a taskitem is
+
+
    +
  • a small implementation work packet
    • +
  • title, requirements, constraints
    • +
  • acceptance criteria and dependencies
    • +
+
+
20work packet
+
+ +
+
Taskitems need prep decisions
+
+

What has to happen before this task runs?

+
+
21the gate question
+
+ +
+
The prep checklist
+
+
    +
  • validate the intake
    • +
  • resolve dependencies
    • +
  • send to architect review
    • +
+
+
22small yes/no decisions
+
+ +
+
In the repo
+
+

prereq_op_selector

+

picks the prerequisite operations for a taskitem

+
+
23technical name
+
+ +
+
Tiny transformer gate
+
+

Learn the messy input.

+

Keep the output menu fixed.

+
+
24middle path
+
+ +
+
Actual training cost
+
+
+
3 to 5minutes
+
3060RTX GPU
+
+

per transformer gate

+
+
25consumer hardware
+
+ +
+
Local gate cost shape
+
+ 
10 planning gates
+x 3 to 5 minutes each
+= ~30 to 50 minutes one-time training
+
+then local inference
+
+
26train once, reuse
+
+ +
+
Decision runtime intuition
+
+ 
training speed proxy:
+4000 steps / 3 to 5 min
+= ~13 to 22 steps/sec
+
+1,650 planning decisions
+= minutes, not token invoices
+
+
27pessimistic proxy
+
+ +
+
Actual model scale
+
+
+
213Kparameters
+
800KBcheckpoint
+
+

one specialist per small gate

+
+
26the whole model is small
+
+ +
+
Training data source
+
+

LLM as scaffolding.

+

Use it to bootstrap examples, not to sit in every runtime decision.

+
+
27distill repeated decisions
+
+ +
+
Synthetic data can help
+
+
    +
  • generate taskitem variants
    • +
  • assign known labels
    • +
  • oversample edge cases
    • +
  • train the tiny gate directly
    • +
+
+
28cheap dataset iteration
+
+ +
+
First extracted dataset
+
+
+
1297clean rows
+
1102train
+
195eval
+
+
+
29real pipeline artifacts
+
+ +
+
The data caught a design mistake
+
+

needs_validate_intake

+

was always true: 1297 / 1297

+
+
30not a learned decision
+
+ +
+
Split the gate
+
+
+
Gate 1
resolve dependencies?
+
Gate 2
architect review?
+
+
+
31smaller decisions
+
+ +
+
First results
+
+ + + + +
GateAccuracyTraining
resolve dependencies69.4%3-5 min on 3060
architect review75.0%3-5 min on 3060
+
+
32diagnostic, not a victory lap
+
+ +
+
What we learned
+
+

1297 rows did not mean 1297 different examples.

+

There were only about 17 unique text templates.

+
+
33row count is not diversity
+
+ +
+
The comparison
+
+ + + + + +
ApproachInput handlingRuntimeOutput control
Regexmanual enumerationtinyexcellent
LLM / SLM + LoRAeasybase model still runsneeds guardrails
Tiny gate transformerlearned~800KB specialistfixed output head
+
+
34the middle path
+
+ +
+
Final claim
+
+
+
AST-first gets the program out fast.
+
RSA makes the repeated planning local.
+
+
+
35small menu, small machine
+
+ +
+
Number sources
+
+
    +
  • taskitem counts: extracted WhetstoneRSA rows
    • +
  • 10+ decisions/taskitem: pipeline audit
    • +
  • 3 to 5 min/gate: RTX 3060 training run
    • +
  • API prices: OpenAI model docs, Apr 2026
    • +
+
+
36measured vs estimated
+
+ +
+
End state
+
+

Product spec in.

+

Program out.

+

Faster, cheaper, more predictable each time the gates move local.

+
+
36the product promise
+
+ +
+ +
Click / arrows: navigate · N: notes · F: fullscreen
+
+
+ + + + diff --git a/presentations/whetstoneRSA_overkill_deck.html b/presentations/whetstoneRSA_overkill_deck.html new file mode 100644 index 0000000..0732e6a --- /dev/null +++ b/presentations/whetstoneRSA_overkill_deck.html @@ -0,0 +1,725 @@ + + + + + +WhetstoneRSA - When LLMs Are Overkill + + + +
+ +
+
WhetstoneRSA
+
+

When LLMs Are Overkill

+

Messy input, small menu, local transformer gate

+
+
1main claim
+
+ +
+
The starting point
+
+

LLMs are amazing.

+

But many software decisions are much smaller than an LLM.

+
+
2not anti-LLM
+
+ +
+
The shape
+
+
The input can be huge.
+
The output is small.
+
+
3the key distinction
+
+ +
+
Example request
+
+

show me the failed builds from yesterday

+
+
4messy human phrasing
+
+ +
+
Same intent, many phrasings
+
+
    +
  • what broke in CI yesterday
    • +
  • pull up yesterday's red builds
    • +
  • did the pipeline fail last night
    • +
  • which build jobs failed since yesterday
    • +
+
+
5input variety
+
+ +
+
What the app actually needs
+
+
+
action
search_builds
+
status
failed
+
date
yesterday
+
+
+
6small structured output
+
+ +
+
Output scale
+
+ 
5 actions x 4 statuses x 4 date buckets
+= about 80 structured outputs
+
+
7small menu
+
+ +
+
Classical code starts well
+
+ 
if text has "failed" and "build":
+    action = search_builds
+
+if text has "yesterday":
+    date = yesterday
+
+
8simple at first
+
+ +
+
Then regex pays the human tax
+
+

You enumerate the input side.

+

Every new phrase becomes another pattern, exception, or bug.

+
+
9input enumeration
+
+ +
+
Rule explosion
+
+ 
5 actions
+x 10 common phrasings
+x 4 statuses
+x 8 time phrasings
+= 1,600-ish rule cases
+
+
10conservative example
+
+ +
+
Regex tradeoff
+
+
    +
  • excellent output control
    • +
  • tiny runtime cost
    • +
  • input coverage gets expensive
    • +
+
+
11great gate, painful input
+
+ +
+
LLMs feel easy
+
+

They understand the messy input.

+

One prompt can replace a pile of brittle phrase rules.

+
+
12the appeal is real
+
+ +
+
But the scale is wrong
+
+

A giant general model

+

choosing from a tiny application menu.

+
+
13overkill
+
+ +
+
API runtime scale
+
+ + + + + +
ModelInputOutput
GPT-4.1$2 / 1M tokens$8 / 1M tokens
GPT-4.1 mini$0.40 / 1M tokens$1.60 / 1M tokens
GPT-4.1 nano$0.10 / 1M tokens$0.40 / 1M tokens
+
+
14OpenAI pricing, checked Apr 2026
+
+ +
+
Fine-tuning scale
+
+ + + + +
Fine-tuned modelTrainingRuntime inputRuntime output
GPT-4.1 mini$5 / 1M tokens$0.80 / 1M$3.20 / 1M
GPT-4.1 nano$1.50 / 1M tokens$0.20 / 1M$0.80 / 1M
+
+
15fine-tuned runtime still has a meter
+
+ +
+
My original plan
+
+

Use LoRA on a local SLM.

+

Adapt a small language model to WhetstoneDSL decisions.

+
+
16the first instinct
+
+ +
+
LoRA still carries the base model
+
+
    +
  • 1B / 3B / 7B parameter base
    • +
  • adapter is small
    • +
  • runtime model is still big
    • +
+
+
17adapter small, base still large
+
+ +
+
The better question
+
+

Was the task ever large enough to deserve that model?

+
+
18question the premise
+
+ +
+
Where this came from
+
+

WhetstoneDSL

+

AST-first programming and universal transpiling

+
+
19creation story
+
+ +
+
What a taskitem is
+
+
    +
  • a small implementation work packet
    • +
  • title, requirements, constraints
    • +
  • acceptance criteria and dependencies
    • +
+
+
20work packet
+
+ +
+
Taskitems need prep decisions
+
+

What has to happen before this task runs?

+
+
21the gate question
+
+ +
+
The prep checklist
+
+
    +
  • validate the intake
    • +
  • resolve dependencies
    • +
  • send to architect review
    • +
+
+
22small yes/no decisions
+
+ +
+
In the repo
+
+

prereq_op_selector

+

picks the prerequisite operations for a taskitem

+
+
23technical name
+
+ +
+
Tiny transformer gate
+
+

Learn the messy input.

+

Keep the output menu fixed.

+
+
24middle path
+
+ +
+
Actual training cost
+
+
+
3 to 5minutes
+
3060RTX GPU
+
+

per transformer gate

+
+
25consumer hardware
+
+ +
+
Actual model scale
+
+
+
213Kparameters
+
800KBcheckpoint
+
+

one specialist per small gate

+
+
26the whole model is small
+
+ +
+
Training data source
+
+

LLM as scaffolding.

+

Use it to bootstrap examples, not to sit in every runtime decision.

+
+
27distill repeated decisions
+
+ +
+
Synthetic data can help
+
+
    +
  • generate taskitem variants
    • +
  • assign known labels
    • +
  • oversample edge cases
    • +
  • train the tiny gate directly
    • +
+
+
28cheap dataset iteration
+
+ +
+
First extracted dataset
+
+
+
1297clean rows
+
1102train
+
195eval
+
+
+
29real pipeline artifacts
+
+ +
+
The data caught a design mistake
+
+

needs_validate_intake

+

was always true: 1297 / 1297

+
+
30not a learned decision
+
+ +
+
Split the gate
+
+
+
Gate 1
resolve dependencies?
+
Gate 2
architect review?
+
+
+
31smaller decisions
+
+ +
+
First results
+
+ + + + +
GateAccuracyTraining
resolve dependencies69.4%3-5 min on 3060
architect review75.0%3-5 min on 3060
+
+
32diagnostic, not a victory lap
+
+ +
+
What we learned
+
+

1297 rows did not mean 1297 different examples.

+

There were only about 17 unique text templates.

+
+
33row count is not diversity
+
+ +
+
The comparison
+
+ + + + + +
ApproachInput handlingRuntimeOutput control
Regexmanual enumerationtinyexcellent
LLM / SLM + LoRAeasybase model still runsneeds guardrails
Tiny gate transformerlearned~800KB specialistfixed output head
+
+
34the middle path
+
+ +
+
Final claim
+
+
+
LoRA adapts a large model to the task.
+
WhetstoneRSA asks whether the task deserved the large model.
+
+
+
35small menu, small machine
+
+ +
+ +
Click / arrows: navigate · N: notes · F: fullscreen
+
+
+ + + +