Initial whetstone_RSA architecture and C++ scaffold
This commit is contained in:
Bill Holcombe
223
docs/whetstone_ast_first_integration.md
Normal file
223
docs/whetstone_ast_first_integration.md
Normal file
@@ -0,0 +1,223 @@
|
||||
# Whetstone-AST-First Integration
|
||||
|
||||
## Purpose
|
||||
|
||||
This document defines how `whetstone_RSA` should relate to `whetstone_DSL`.
|
||||
|
||||
The key rule is:
|
||||
|
||||
`whetstone_RSA` should be Whetstone-AST-first, not merely AST-first.
|
||||
|
||||
That means the semantic source of truth for RSA contracts should live in the
|
||||
Whetstone AST and annotation system, not in ad hoc C++ structs or generic parser
|
||||
ASTs.
|
||||
|
||||
## Why Generic AST Is Not Enough
|
||||
|
||||
A generic AST is mostly syntax structure.
|
||||
|
||||
It can tell you things like:
|
||||
|
||||
- function
|
||||
- class
|
||||
- parameter
|
||||
- if-statement
|
||||
- call expression
|
||||
|
||||
That is useful, but it is not enough for RSA.
|
||||
|
||||
RSA needs first-class semantic contracts for things like:
|
||||
|
||||
- input structure
|
||||
- input modality
|
||||
- output topology
|
||||
- output constraint
|
||||
- policy stability
|
||||
- suitability assessment
|
||||
- gate shape
|
||||
- confidence behavior
|
||||
- abstain policy
|
||||
- escalation policy
|
||||
- diagnosis metadata
|
||||
|
||||
Those are not reliable consequences of syntax alone.
|
||||
They are authored semantic data.
|
||||
|
||||
## Why Whetstone AST Is The Right Source
|
||||
|
||||
The Whetstone AST already exists to carry language-neutral semantic intent beyond
|
||||
what ordinary source code or parser ASTs can express cleanly.
|
||||
|
||||
That makes it the right authoring substrate for RSA because:
|
||||
|
||||
- annotations are first-class
|
||||
- concepts are language-neutral
|
||||
- contracts can be preserved across projections
|
||||
- comments do not need to carry architectural meaning
|
||||
- generated C++ can remain a projection target instead of becoming the semantic home
|
||||
|
||||
In short:
|
||||
|
||||
- generic AST gives structure
|
||||
- Whetstone AST gives structure plus semantic contract capacity
|
||||
|
||||
RSA needs the second one.
|
||||
|
||||
## Architectural Rule
|
||||
|
||||
The semantic source of truth for RSA should live in `whetstone_DSL`.
|
||||
|
||||
The C++ runtime in `whetstone_RSA` should be treated as a projection target for:
|
||||
|
||||
- runtime data types
|
||||
- execution logic
|
||||
- probe registry interfaces
|
||||
- policy and diagnosis runtime behavior
|
||||
|
||||
This keeps the architecture aligned with the rest of the Whetstone program:
|
||||
|
||||
- AST is source of truth
|
||||
- target languages are projections
|
||||
|
||||
## Proposed Split Of Responsibilities
|
||||
|
||||
### `whetstone_DSL`
|
||||
|
||||
Owns:
|
||||
|
||||
- RSA semantic ontology
|
||||
- gate definitions as semantic contracts
|
||||
- suitability metadata
|
||||
- diagnosis metadata
|
||||
- probe declarations
|
||||
- projection rules into target runtimes
|
||||
|
||||
### `whetstone_RSA`
|
||||
|
||||
Owns:
|
||||
|
||||
- C++ runtime implementation
|
||||
- probe execution engine
|
||||
- diagnosis engine
|
||||
- policy runtime
|
||||
- compiled/runtime-facing representations of projected contracts
|
||||
|
||||
## What Should Exist In The Whetstone AST
|
||||
|
||||
RSA needs explicit semantic objects, not just ordinary code nodes.
|
||||
|
||||
Examples of first-class RSA concepts that should eventually exist in Whetstone form:
|
||||
|
||||
- `GateDefinition`
|
||||
- `GateEvidence`
|
||||
- `GateDiagnosis`
|
||||
- `GateSuitability`
|
||||
- `PolicyRecommendation`
|
||||
- `ProbeDefinition`
|
||||
- `DecisionContract`
|
||||
|
||||
These should be represented as semantic entities with structured fields, not as
|
||||
freeform comments embedded in generated C++.
|
||||
|
||||
## Why This Is Better Than Authoring In C++
|
||||
|
||||
If RSA contracts are authored directly in C++, the semantics become fragile:
|
||||
|
||||
- comments carry architectural meaning
|
||||
- relationships are implicit
|
||||
- code review sees implementation before contract
|
||||
- projection to other targets becomes harder
|
||||
|
||||
If RSA contracts are authored in the Whetstone AST:
|
||||
|
||||
- the semantics are explicit
|
||||
- annotations stay attached to the real concept
|
||||
- multiple targets can be generated later
|
||||
- runtime code becomes easier to keep honest
|
||||
|
||||
This is especially important because RSA is intended to remain model-agnostic.
|
||||
|
||||
The contract is primary.
|
||||
The current transformer-based runtime is secondary.
|
||||
|
||||
## Model-Agnostic Benefit
|
||||
|
||||
Because RSA is contractual rather than model-defined, the AST layer should describe:
|
||||
|
||||
- what the gate promises
|
||||
- what the gate consumes
|
||||
- how confidence and abstention behave
|
||||
- how diagnosis works
|
||||
|
||||
without hard-coding one backend.
|
||||
|
||||
That preserves future replacement paths such as:
|
||||
|
||||
- transformer specialist
|
||||
- classical ML classifier
|
||||
- RL policy
|
||||
- retrieval-backed scorer
|
||||
- deterministic-plus-learned hybrid
|
||||
|
||||
The Whetstone AST should capture the contract.
|
||||
The C++ runtime should implement one or more backends against it.
|
||||
|
||||
## Near-Term Practical Rule
|
||||
|
||||
Right now, some C++ scaffolding already exists in `whetstone_RSA`.
|
||||
|
||||
That is fine as a bootstrap.
|
||||
|
||||
But it should be treated as:
|
||||
|
||||
- provisional runtime code
|
||||
- useful for exercising the contract shape
|
||||
- not the long-term semantic source of truth
|
||||
|
||||
As soon as the Whetstone-side semantic representation exists, the C++ layer should
|
||||
be aligned to it rather than continuing as an independently evolving hand-authored
|
||||
model.
|
||||
|
||||
## How To Use WhetstoneDSL Here
|
||||
|
||||
The right use of `whetstone_DSL` is not "parse some C++ and hope the meaning falls
|
||||
out."
|
||||
|
||||
The right use is:
|
||||
|
||||
1. define RSA concepts explicitly in Whetstone semantic form
|
||||
2. attach first-class metadata and annotations there
|
||||
3. project those contracts into C++ runtime types and adapters
|
||||
|
||||
If existing Whetstone MCP tools can help bootstrap some of the current C++ structs
|
||||
into AST-native representations, that is useful.
|
||||
|
||||
But the goal is not mechanical conversion for its own sake.
|
||||
The goal is semantic alignment.
|
||||
|
||||
If direct manual authoring in Whetstone form lands closer to the intended contract,
|
||||
that is better than a weak automatic conversion.
|
||||
|
||||
## Practical Guidance For The Current Stage
|
||||
|
||||
At this stage of the project:
|
||||
|
||||
- keep the current C++ runtime scaffold because it is useful for validating runtime shape
|
||||
- do not expand C++ into the permanent semantic home of the system
|
||||
- prioritize defining the RSA domain model in Whetstone semantic terms
|
||||
- use MCP-assisted conversion only when it lands close to the intended semantic contract
|
||||
|
||||
This is still early enough that correctness of semantic shape matters more than
|
||||
automation of the conversion path.
|
||||
|
||||
## Implication
|
||||
|
||||
`whetstone_RSA` is not just "a C++ library with some docs."
|
||||
|
||||
It should become:
|
||||
|
||||
- a Whetstone-authored semantic subsystem
|
||||
- with a C++ runtime projection
|
||||
|
||||
That is the architecture most consistent with both Whetstone and the long-term
|
||||
goals of RSA.
|
||||
Reference in New Issue
Block a user