WIP: save progress 2026-04-12
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Bill Holcombe
@@ -221,3 +221,652 @@ It should become:
|
||||
|
||||
That is the architecture most consistent with both Whetstone and the long-term
|
||||
goals of RSA.
|
||||
|
||||
## Bootstrap Mapping From The Current C++ Scaffold
|
||||
|
||||
The current C++ code is useful bootstrap material, but it should now be treated as
|
||||
input to the semantic model rather than the long-term definition of that model.
|
||||
|
||||
This section inventories the current scaffold and classifies each piece as either:
|
||||
|
||||
- semantic contract that should move into Whetstone-authored form
|
||||
- runtime behavior that should remain in projected C++
|
||||
- bootstrap/demo code that should not define long-term architecture
|
||||
|
||||
### Current C++ Semantic Surface
|
||||
|
||||
From `include/whetstone_rsa/types.h`, the current scaffold already expresses the
|
||||
core semantic vocabulary that should be re-authored in Whetstone form:
|
||||
|
||||
- ontology enums
|
||||
- `InputStructure`
|
||||
- `InputModality`
|
||||
- `OutputTopology`
|
||||
- `OutputConstraint`
|
||||
- `LabelStability`
|
||||
- `TargetLayer`
|
||||
- diagnosis enums
|
||||
- `FailureClass`
|
||||
- `Intervention`
|
||||
- `DeployMode`
|
||||
- `RiskTier`
|
||||
- contract entities
|
||||
- `GateDefinition`
|
||||
- `GateEvidence`
|
||||
- `PolicyContext`
|
||||
- `ProbeRequest`
|
||||
- `ProbeResult`
|
||||
- `GateDiagnosis`
|
||||
- `GateSuitability`
|
||||
- `PolicyRecommendation`
|
||||
|
||||
These should not remain primarily hand-authored C++ concepts.
|
||||
|
||||
They should become semantic entities in the Whetstone source of truth.
|
||||
|
||||
### Recommended Whetstone Semantic Entities
|
||||
|
||||
The current scaffold suggests the following first semantic model.
|
||||
|
||||
#### 1. `DecisionContract`
|
||||
|
||||
This should become the semantic home for the ontology-level fields that currently
|
||||
sit inside `GateDefinition`.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
- `gate_id`
|
||||
- `task_family`
|
||||
- `input_structure`
|
||||
- `input_modality`
|
||||
- `output_topology`
|
||||
- `output_constraint`
|
||||
- `risk_tier`
|
||||
- `policy_stability`
|
||||
- `deterministic_baseline_available`
|
||||
- `supports_abstain`
|
||||
- `metadata`
|
||||
|
||||
This is the contract boundary, independent of backend family.
|
||||
|
||||
#### 2. `DecisionSurface`
|
||||
|
||||
The current `GateDefinition.labels` plus `candidate_factorizations` really describe
|
||||
the shape of the output space, not just the gate identity.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
- `output_labels`
|
||||
- `slot_schema`
|
||||
- `candidate_factorizations`
|
||||
- `candidate_gate_shapes`
|
||||
- `invalid_combinations`
|
||||
- `deterministic_prepass_options`
|
||||
|
||||
This gives the AST layer a place to describe flat multiclass, hierarchical, or
|
||||
factorized designs without forcing that structure to remain implicit in C++.
|
||||
|
||||
#### 3. `DecisionRequestModel`
|
||||
|
||||
The current C++ scaffold does not yet have a first-class request packet type
|
||||
separate from evaluation/demo inputs.
|
||||
|
||||
That should be corrected in the AST-first model.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
- `packet_id`
|
||||
- `text_fields`
|
||||
- `symbolic_fields`
|
||||
- `numeric_fields`
|
||||
- `context_fields`
|
||||
- `state_fields`
|
||||
- `provenance`
|
||||
|
||||
This entity should support both enum-style gates and bounded structured outputs.
|
||||
|
||||
#### 4. `DecisionResultModel`
|
||||
|
||||
The current scaffold implicitly assumes label selection and policy behavior, but the
|
||||
AST-first model should make result structure explicit.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
- `decision_id`
|
||||
- `selected_label`
|
||||
- `filled_slots`
|
||||
- `confidence`
|
||||
- `abstain`
|
||||
- `retry_strategy`
|
||||
- `escalation_target`
|
||||
- `trace_packet`
|
||||
|
||||
This should later project into both C++ runtime structs and packaging artifacts.
|
||||
|
||||
#### 5. `DiagnosisModel`
|
||||
|
||||
The current `GateDiagnosis` is already close to a semantic object and should be
|
||||
preserved almost directly, but moved into Whetstone-authored form.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
- `primary_failure_class`
|
||||
- `secondary_failure_classes`
|
||||
- `supporting_signals`
|
||||
- `recommended_probes`
|
||||
- `recommended_interventions`
|
||||
- `recommended_policy`
|
||||
- `confidence`
|
||||
|
||||
#### 6. `ProbeDefinition`
|
||||
|
||||
The current `Probe` interface and `ProbeRequest` / `ProbeResult` pair indicate that
|
||||
probes are part of the semantic model, not just implementation detail.
|
||||
|
||||
The AST layer should explicitly represent:
|
||||
|
||||
- `probe_id`
|
||||
- `applies_to_gate_shapes`
|
||||
- `required_evidence_fields`
|
||||
- `produced_signals`
|
||||
- `failure_hints`
|
||||
- `intervention_hints`
|
||||
|
||||
Concrete runtime probes such as `confidence_threshold_probe` should then be C++
|
||||
implementations of these declared probe contracts.
|
||||
|
||||
#### 7. `SuitabilityAssessment`
|
||||
|
||||
The current `GateSuitability` type should become a semantic output model for the
|
||||
pre-RSA decision that determines whether the problem belongs in the RSA layer at all.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
- `recommended`
|
||||
- `target_layer`
|
||||
- `rationale`
|
||||
- `bounded_output_confidence`
|
||||
- `deterministic_baseline_available`
|
||||
- `label_stability_risk`
|
||||
- `suggested_gate_shape`
|
||||
|
||||
#### 8. `BackendSuitability`
|
||||
|
||||
This does not yet exist in code, but the roadmap now requires it.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
- `gate_id`
|
||||
- `candidate_backends`
|
||||
- `recommended_backend`
|
||||
- `rationale`
|
||||
- `confidence`
|
||||
- `benchmark_refs`
|
||||
|
||||
This must be semantic-first, not added ad hoc in C++ later.
|
||||
|
||||
#### 9. `GatePackage`
|
||||
|
||||
The runtime needs a durable packaged form for serving decisions after offline
|
||||
selection and calibration.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
- `gate_id`
|
||||
- `contract_ref`
|
||||
- `gate_shape`
|
||||
- `backend_family`
|
||||
- `artifact_ref`
|
||||
- `thresholds`
|
||||
- `abstain_policy`
|
||||
- `escalation_policy`
|
||||
- `trace_schema`
|
||||
- `version`
|
||||
|
||||
### What Should Remain Runtime-Only In C++
|
||||
|
||||
The following concepts should remain projected runtime behavior rather than become
|
||||
the semantic source of truth:
|
||||
|
||||
- `Probe` virtual interface
|
||||
- `ProbeRegistry`
|
||||
- `DiagnosisEngine`
|
||||
- `SuitabilityAssessor`
|
||||
- backend adapter implementations
|
||||
- calibration code
|
||||
- packaging loaders
|
||||
- benchmark runners
|
||||
|
||||
These are execution mechanisms, not the semantic model itself.
|
||||
|
||||
### What In The Current Scaffold Is Only Bootstrap
|
||||
|
||||
The following parts are useful but should not steer the long-term semantic design:
|
||||
|
||||
- `examples/diagnose_demo.cpp`
|
||||
- the current hand-authored stringification helpers
|
||||
- any current field arrangement that exists only because it was convenient for the
|
||||
first C++ scaffold
|
||||
|
||||
### Immediate Migration Map
|
||||
|
||||
The practical next move should be:
|
||||
|
||||
1. Represent the current C++ contract entities above as Whetstone semantic objects.
|
||||
2. Mark which current fields are semantic contract versus runtime-only detail.
|
||||
3. Treat `types.h` as a temporary projected contract snapshot, not the authoring
|
||||
home of the model.
|
||||
4. Add projection rules so future C++ runtime types are derived from the semantic
|
||||
source rather than independently evolved.
|
||||
|
||||
### Most Important Boundary To Preserve
|
||||
|
||||
The current scaffold already mixes two different layers that must stay separate:
|
||||
|
||||
- semantic contract
|
||||
- runtime execution strategy
|
||||
|
||||
If that separation is preserved now, `whetstone_RSA` can become genuinely AST-first.
|
||||
If it is not preserved, the project will keep drifting into a C++-first runtime
|
||||
with AST plans attached afterward.
|
||||
|
||||
## First RSA Semantic Model
|
||||
|
||||
This section defines the first concrete semantic model that should become the
|
||||
source of truth for `whetstone_RSA`.
|
||||
|
||||
It is intentionally small and close to the current scaffold so migration can
|
||||
start immediately without pretending the architecture is already finished.
|
||||
|
||||
The design rule is:
|
||||
|
||||
- semantic entities describe contract and meaning
|
||||
- projected C++ types describe runtime execution
|
||||
|
||||
### Model Goals
|
||||
|
||||
The first semantic model must support all of the following from the beginning:
|
||||
|
||||
- bounded enum decisions
|
||||
- bounded structured slot-filling decisions
|
||||
- suitability assessment before a gate is adopted
|
||||
- diagnosis after a gate underperforms
|
||||
- gate-shape comparison
|
||||
- backend-selection metadata
|
||||
- runtime packaging metadata
|
||||
|
||||
### Semantic Entity Set
|
||||
|
||||
The first semantic model should consist of these entities:
|
||||
|
||||
- `DecisionContract`
|
||||
- `DecisionSurface`
|
||||
- `DecisionRequestModel`
|
||||
- `DecisionResultModel`
|
||||
- `SuitabilityAssessment`
|
||||
- `DiagnosisModel`
|
||||
- `ProbeDefinition`
|
||||
- `BackendSuitability`
|
||||
- `GatePackage`
|
||||
|
||||
### 1. `DecisionContract`
|
||||
|
||||
This is the semantic identity of an RSA gate.
|
||||
|
||||
It answers:
|
||||
|
||||
- what decision surface exists
|
||||
- what kind of inputs it consumes
|
||||
- what kind of bounded outputs it promises
|
||||
- what policy constraints govern deployment
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
DecisionContract
|
||||
gate_id
|
||||
task_family
|
||||
input_structure
|
||||
input_modality
|
||||
output_topology
|
||||
output_constraint
|
||||
risk_tier
|
||||
policy_stability
|
||||
deterministic_baseline_available
|
||||
supports_abstain
|
||||
metadata
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `policy_stability` should replace C++ naming that implies only labels drift.
|
||||
- this entity should not include backend-family decisions
|
||||
- this entity should not include benchmark evidence
|
||||
|
||||
### 2. `DecisionSurface`
|
||||
|
||||
This is the semantic description of the output space.
|
||||
|
||||
It is separate from `DecisionContract` because two gates can share similar
|
||||
contract properties but differ in shape:
|
||||
|
||||
- flat enum choice
|
||||
- factorized decision
|
||||
- hierarchical route
|
||||
- bounded slot fill
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
DecisionSurface
|
||||
gate_id
|
||||
primary_mode // enum | structured | hybrid
|
||||
output_labels
|
||||
slot_schema
|
||||
candidate_gate_shapes // binary | multiclass | factorized | hierarchical
|
||||
candidate_factorizations
|
||||
invalid_combinations
|
||||
deterministic_prepass_options
|
||||
downstream_executor_kind
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- `slot_schema` is required from version 0 even if many early gates are enum-only
|
||||
- `invalid_combinations` should make structured outputs safe by contract
|
||||
- `downstream_executor_kind` should record what deterministic system consumes the result
|
||||
|
||||
### 3. `DecisionRequestModel`
|
||||
|
||||
This is the semantic input packet model for runtime use.
|
||||
|
||||
It should support hybrid packets directly instead of forcing a project to flatten
|
||||
everything into one string.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
DecisionRequestModel
|
||||
packet_id
|
||||
gate_id
|
||||
text_fields
|
||||
symbolic_fields
|
||||
numeric_fields
|
||||
context_fields
|
||||
state_fields
|
||||
provenance
|
||||
```
|
||||
|
||||
Expected usage:
|
||||
|
||||
- `text_fields`
|
||||
- natural language summaries
|
||||
- extracted comments
|
||||
- short requirement packets
|
||||
- `symbolic_fields`
|
||||
- AST node kinds
|
||||
- annotations
|
||||
- enums
|
||||
- dependency graphs
|
||||
- `numeric_fields`
|
||||
- counts
|
||||
- scores
|
||||
- ratios
|
||||
- latency or complexity features
|
||||
|
||||
### 4. `DecisionResultModel`
|
||||
|
||||
This is the canonical semantic output model for runtime decisions.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
DecisionResultModel
|
||||
decision_id
|
||||
gate_id
|
||||
selected_label
|
||||
filled_slots
|
||||
confidence
|
||||
abstain
|
||||
retry_strategy
|
||||
escalation_target
|
||||
trace_packet
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- enum gates mainly use `selected_label`
|
||||
- structured gates mainly use `filled_slots`
|
||||
- hybrid outputs may use both
|
||||
|
||||
### 5. `SuitabilityAssessment`
|
||||
|
||||
This is the semantic result of asking whether a problem belongs in RSA at all.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
SuitabilityAssessment
|
||||
gate_id
|
||||
recommended
|
||||
target_layer // deterministic | rsa | slm | llm | human
|
||||
rationale
|
||||
bounded_output_confidence
|
||||
deterministic_baseline_available
|
||||
policy_stability_risk
|
||||
suggested_gate_shape
|
||||
```
|
||||
|
||||
This entity is intentionally separate from diagnosis and backend selection.
|
||||
|
||||
### 6. `DiagnosisModel`
|
||||
|
||||
This is the semantic result of evaluating a weak or risky gate.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
DiagnosisModel
|
||||
gate_id
|
||||
primary_failure_class
|
||||
secondary_failure_classes
|
||||
supporting_signals
|
||||
recommended_probes
|
||||
recommended_interventions
|
||||
recommended_policy
|
||||
confidence
|
||||
```
|
||||
|
||||
This should remain close to the current C++ `GateDiagnosis`.
|
||||
|
||||
### 7. `ProbeDefinition`
|
||||
|
||||
This is the semantic declaration of a diagnostic probe.
|
||||
|
||||
The semantic layer should define what a probe means before runtime C++ implements
|
||||
how it executes.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
ProbeDefinition
|
||||
probe_id
|
||||
applies_to_gate_shapes
|
||||
required_contract_fields
|
||||
required_evidence_fields
|
||||
produced_signals
|
||||
failure_hints
|
||||
intervention_hints
|
||||
```
|
||||
|
||||
Concrete examples:
|
||||
|
||||
- `confidence_threshold_probe`
|
||||
- `class_confusion_probe`
|
||||
- `factorization_probe`
|
||||
- `deterministic_rule_probe`
|
||||
- `context_width_probe`
|
||||
|
||||
### 8. `BackendSuitability`
|
||||
|
||||
This is the semantic result of backend comparison after a gate has already been
|
||||
judged to belong in RSA.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
BackendSuitability
|
||||
gate_id
|
||||
candidate_backends
|
||||
recommended_backend
|
||||
rationale
|
||||
confidence
|
||||
benchmark_refs
|
||||
```
|
||||
|
||||
Expected backend families:
|
||||
|
||||
- `linear_model`
|
||||
- `gradient_boosting`
|
||||
- `random_forest`
|
||||
- `bayesian`
|
||||
- `transformer`
|
||||
- `hybrid`
|
||||
- `rl_policy`
|
||||
|
||||
`rl_policy` should exist in the model now even if it remains an unimplemented
|
||||
runtime placeholder for the first version.
|
||||
|
||||
### 9. `GatePackage`
|
||||
|
||||
This is the semantic deployment artifact description.
|
||||
|
||||
It exists so runtime loading is a projection of semantic packaging decisions,
|
||||
not a one-off C++ concern.
|
||||
|
||||
Proposed fields:
|
||||
|
||||
```text
|
||||
GatePackage
|
||||
gate_id
|
||||
contract_ref
|
||||
surface_ref
|
||||
gate_shape
|
||||
backend_family
|
||||
artifact_ref
|
||||
thresholds
|
||||
abstain_policy
|
||||
escalation_policy
|
||||
trace_schema
|
||||
version
|
||||
```
|
||||
|
||||
### Semantic Relationships
|
||||
|
||||
The first model should follow these relationships:
|
||||
|
||||
```text
|
||||
DecisionContract
|
||||
owns high-level decision identity
|
||||
|
||||
DecisionSurface
|
||||
refines output shape for a contract
|
||||
|
||||
DecisionRequestModel
|
||||
is evaluated against DecisionContract + DecisionSurface
|
||||
|
||||
DecisionResultModel
|
||||
must satisfy DecisionSurface constraints
|
||||
|
||||
SuitabilityAssessment
|
||||
determines whether the decision belongs in RSA at all
|
||||
|
||||
DiagnosisModel
|
||||
explains weak performance or risky deployment
|
||||
|
||||
ProbeDefinition
|
||||
declares reusable diagnosis mechanisms
|
||||
|
||||
BackendSuitability
|
||||
recommends which implementation family should fill an RSA contract
|
||||
|
||||
GatePackage
|
||||
records the chosen runtime projection of a validated contract
|
||||
```
|
||||
|
||||
### Projection Map To The Current C++ Scaffold
|
||||
|
||||
The current scaffold should be treated as the first runtime projection target.
|
||||
|
||||
#### Project directly from semantics into current C++ types
|
||||
|
||||
```text
|
||||
DecisionContract -> GateDefinition core fields
|
||||
DecisionSurface -> GateDefinition.labels + candidate_factorizations + future slot fields
|
||||
DecisionResultModel -> future runtime result structs
|
||||
SuitabilityAssessment -> GateSuitability
|
||||
DiagnosisModel -> GateDiagnosis
|
||||
ProbeDefinition -> Probe metadata + ProbeRequest/ProbeResult contracts
|
||||
```
|
||||
|
||||
#### Runtime-only C++ types that should remain projected
|
||||
|
||||
```text
|
||||
Probe
|
||||
ProbeRegistry
|
||||
DiagnosisEngine
|
||||
SuitabilityAssessor
|
||||
ConfidenceThresholdProbe
|
||||
```
|
||||
|
||||
These are implementations, not semantic source objects.
|
||||
|
||||
### Immediate Type Migration Rule
|
||||
|
||||
Until semantic authoring is fully in place, the current C++ fields should be
|
||||
interpreted according to this split:
|
||||
|
||||
- semantic-now
|
||||
- ontology enums
|
||||
- gate contract fields
|
||||
- diagnosis enums
|
||||
- suitability result fields
|
||||
- runtime-now
|
||||
- probe interfaces
|
||||
- engine wiring
|
||||
- confidence-threshold implementation
|
||||
- demo executable
|
||||
|
||||
When a new field is proposed, the default question should be:
|
||||
|
||||
- does this describe semantic meaning?
|
||||
- if yes, add it to the semantic model first
|
||||
- does this describe runtime execution detail?
|
||||
- if yes, keep it in projected C++
|
||||
|
||||
### Immediate Next Conversion Step
|
||||
|
||||
The next concrete implementation step after this document should be:
|
||||
|
||||
1. represent `DecisionContract`, `DecisionSurface`, and `ProbeDefinition` as the
|
||||
first Whetstone-authored RSA semantic forms
|
||||
2. map each field in `types.h` into one of those forms
|
||||
3. mark missing fields needed for structured slot outputs and backend selection
|
||||
4. treat `types.h` as a temporary projection snapshot until semantic-first
|
||||
generation replaces hand editing
|
||||
|
||||
## Bootstrap Artifact
|
||||
|
||||
The first machine-readable semantic bootstrap artifact now lives at:
|
||||
|
||||
- `semantic/rsa_semantic_bootstrap_v0.json`
|
||||
|
||||
It is intentionally limited to the first three semantic entities needed to begin
|
||||
realignment:
|
||||
|
||||
- `DecisionContract`
|
||||
- `DecisionSurface`
|
||||
- `ProbeDefinition`
|
||||
|
||||
That file should be treated as the current semantic bootstrap source ahead of any
|
||||
further handwritten expansion of C++ contract types.
|
||||
|
||||
Reference in New Issue
Block a user