11 KiB
[Derived:UserConversation-Bill] Semantic Hash as Deterministic Generation Contract
Status: Proposed — pre-sprint Filed: 2026-02-24 Related sprints: Post-130
Ultimate Goal
Eliminate the LLM from the code generation hot path entirely.
The current model is: fuzzy actor synthesizes code from intent at generation time. The target model is: fuzzy actor captures intent as a hash at authoring time; generation is a deterministic lookup.
This is not a distant aspiration — it is the logical extension of what Whetstone already does.
The AST-first architecture, the SemAnno annotation vocabulary, LegalOperationGraph, and
SemanticCoreIR are already building toward a world where a "fuzzy actor" means choosing
from a known set of legal options, not free-form synthesis. The semantic hash is the mechanism
that makes that transition explicit and machine-enforced.
When a node has a hash, the generator does not ask "what should this be?" It asks "do I have a verified template for this hash?" If yes: inflate deterministically. If no: escalate to the SLM, generate once, store against the hash, never generate again.
Core Concept: The Hash IS the Contract
Each AST node carries a semantic_hash that is a canonical fingerprint of its
semantic identity — what it is, not what it's called.
- Human-facing: developers and the editor work with labels (
userBalance,processJob). - System-facing: the generator, validator, and template library work with the hash.
- Cross-language identity: the same hash resolves to valid code in C++, Rust, Python, Go, etc. The language is a rendering target, not the source of truth.
The hash is a pure function of a node's semantic attributes as expressed through the SemAnno vocabulary. It is not a content hash of the source text. Two nodes in different languages with the same semantic contract produce the same hash.
Hash Inputs: Start From What Exists
Do not treat this list as exhaustive. The hash input set should grow as the SemAnno
vocabulary grows. The starting point is the annotation vocabulary already implemented in
CompactAST.h::extractSemanticSummary() — which already aggregates all annotation types
into a canonical JSON object per node.
The hash is computed from a normalized (sorted-key, no-whitespace) serialization of that
summary, plus the node's IRNodeKind from SemanticCoreIR.
Current annotation subjects that contribute to the hash:
| Subject | Annotations | Why They Matter |
|---|---|---|
| Type System | BitWidth, Endian, Layout, Nullability, Variance, Identity, Mutability, TypeState | Define the shape and constraints of values |
| Concurrency | Atomic, Sync, ThreadModel, MemoryBarrier, Exec, Blocking, Parallel | Define threading contract |
| Error Handling | Trap, Exception, Panic | Define the failure contract |
| Scope | Binding, Lookup, Capture, Visibility, Scope | Define lifetime and access |
| Contracts | pre/post conditions, returnShape, sideEffects (from ContractAnnotation) |
The most important inputs for function nodes |
| IR Node Kind | Module / Function / Type / Effect / OwnershipRegion / ConcurrencyRegion | Top-level structural classifier |
| Intent Tags | IRNode::intentTags |
High-level pattern class (pure-math, stateful-actor, io-bridge, etc.) |
Annotations that are not hash inputs (they are metadata about process, not semantic identity):
- Workflow routing annotations (ContextWidth, Review, Priority, ImplementationStatus)
- Optimization hints (Loop, TailCall, Align, Pack) — these are implementation choices, not contracts
- Meta/policy annotations — these describe how to generate, not what to generate
- Human label / name — intentionally excluded; the hash must be name-independent
The right way to discover what should be in the hash: if changing annotation X without changing what the node does would change the hash, it should not be in the hash. If changing annotation X changes the semantic contract, it should be.
Hash Format
H<version>:<hex>
Examples:
H1:8F2A4C9B1E3D7F05(64-bit, 16 hex chars)H1:00000000FFFFFFFF
The H<version> prefix is mandatory from day one. The hash algorithm must be versioned
because any change to the input normalization or hash function invalidates all stored hashes.
A H1: hash and a H2: hash are never compared directly — they come from different
algorithm generations.
Algorithm: canonical JSON serialization (sorted keys, no whitespace) of the semantic attribute set → FNV-1a 64-bit or xxHash64. Either is acceptable; stability is the constraint.
Symbol Table: Hash ↔ Label ↔ Language Syntax
To preserve human readability, a per-project SemanticSymbolTable maps:
H1:8F2A4C9B1E3D7F05 <-> userBalance <-> { cpp: "float", rust: "f32", python: "float" }
This prevents naming drift across transpilation targets and over time. If the same semantic contract is generated in three languages, they share one hash and one symbol table entry. The label is a convenience alias for the human; the hash is the stable identity for the system.
The symbol table is a project-level sidecar (like SemannoSidecar) persisted to disk.
whetstone_assemble_context can query it during context assembly.
Template Library: The Zero-SLM Hot Path
The template library is what makes zero-SLM inflation possible. It is a map:
hash -> verified code fragment (per target language)
When a node needs to be generated:
- Compute the hash from the node's semantic attributes.
- Look up the hash in the template library.
- Hit: emit the stored fragment. No LLM call.
- Miss: call the SLM, generate, verify by re-hashing the result, store if hash matches.
The library starts empty and fills as the project is worked on. Over time, the hit rate increases and LLM calls become rare. Common patterns (pure transforms, typed containers, standard I/O bridges) converge to full coverage quickly.
Re-hashing the generated output as a verification step is the mechanism that prevents "fuzzy error propagation": if the SLM generates code whose semantic contract doesn't match the requested hash, the mismatch is caught immediately at generation time.
Hash Lock Policy
Nodes can be in one of two hash states:
Mutable (default / dev mode): Semantic attributes can change freely. The hash is recomputed on each change. No enforcement is applied. This is the normal authoring state.
Locked (production / API contract):
The hash is pinned. Any mutation that would change the hash triggers a HashLockViolation
diagnostic. This is for nodes whose contract is depended upon by external consumers —
API surface, shared schema types, HiveMind job signatures.
The lock is set explicitly by the developer or by policy (e.g., "lock all nodes marked
@api-surface"). It is not automatically applied. The goal is not to be Rust — it is to
give the developer a physical guardrail on the nodes that actually matter.
Lock state is stored in the symbol table sidecar alongside the hash.
ASTVersionTracker (already in CompactAST.h) already tracks which node IDs changed per
version. The hash lock check integrates naturally: after a mutation, check if any locked
node's hash changed.
Integration Points in Existing Architecture
| Existing Component | Role in Hash System |
|---|---|
SemanticCoreIR |
Provides IRNodeKind and intentTags — top-level hash inputs |
CompactAST::extractSemanticSummary() |
Already aggregates all SemAnno attributes into canonical JSON — this is the hash pre-image |
LegalOperationGraph |
Already models {language, nodeKind} → legal operations — the "legal choices" the fuzzy actor picks from |
ASTVersionTracker |
Already tracks node change sets per version — integrates directly with hash lock checking |
SemannoSidecar |
Pattern for per-project sidecar persistence — reuse for symbol table |
whetstone_validate_taskitem |
Taskitem self-containment score — hash coverage becomes a scoring axis |
whetstone_assemble_context |
Symbol lookup — can return hash alongside label |
whetstone_generate_code |
Generation hot path — hash lookup before SLM call |
ImGui Editor Integration
The hash is an inspector view, not a code annotation. In the editor:
- Semantic Hash Lens: a toggleable overlay in the code panel that shows the hash next to annotated nodes (not in the source text, in the gutter or tooltip).
- Lock indicator: locked nodes get a distinct visual state (color, icon).
- Template library coverage: a panel showing hash hit/miss rates for the current file, as a proxy for "how deterministic is this codebase right now."
- Symbol table browser: a view of
hash ↔ label ↔ language syntaxentries, editable.
Relationship to Taskitem Condensation
A taskitem with a hash is significantly smaller and more machine-actionable than one with a natural language description. The hash encodes the complete semantic contract; the taskitem only needs:
- The hash (what it is)
- The human label (what to call it)
- The target language(s) (where to emit it)
- The dependency edges (what it connects to)
The LLM's role during taskitem authoring is: "given this intent, produce the correct hash." That is a classification problem into a known finite space, not open-ended synthesis. It is orders of magnitude more reliable and cheaper than generating implementation code.
What This Is NOT
- Not a content hash of source code. The hash must be stable across cosmetic changes (reformatting, renaming, comment edits). It is a hash of the semantic contract, derived from SemAnno attributes, not the text.
- Not a replacement for the SLM. The SLM is still needed for the first generation of any hash that isn't in the template library, and for evolving the hash vocabulary itself. The goal is to make each SLM call a one-time investment, not a recurring cost.
- Not enforced everywhere. The policy system determines which nodes are contractual (locked) and which are informative. Most nodes start informative.
Open Questions
-
Hash granularity: should a
Functionnode hash include its parameter hashes, or only its own contract? Including parameter hashes creates a composite/Merkle-style hash that propagates changes upward. Not including them keeps the hash local. Both have uses. -
Side effects in the hash:
ContractAnnotation::sideEffectsis declared intent, not verified behavior. For now, treat it as annotated intent (part of the contract). Static analysis verification is a later concern. -
Hash collision policy: 64-bit address space makes accidental collision astronomically unlikely in practice, but intentional collision is theoretically possible. For now, accept this and treat hashes as trusted within a project.
-
Template library storage format: flat JSON sidecar, SQLite, or content-addressable store? Start with JSON for simplicity, migrate if performance demands it.