whetstone_DSL/CODEGEN_BUG_REPORT.md

generate_code Quality Gap Report

Date: 2026-03-01 Observed by: Claude Code session (Fabricate project, Task 1–2) Binary: editor/build-native/whetstone_mcp (PID 3955, started from /home/bill/Documents) Mode: --language cpp

---

Bug 1: Non-C++ spec returns wrong fallback, gates mislead

Tool: whetstone_generate_code Input spec (verbatim):

Generate a CMakeLists.txt for a CUDA C++17 project called WhetstoneAI_Fabricate targeting
Linux (Ubuntu 24.04), CUDA 13.1, GCC 13, CMake 3.28, GPU architecture sm_86 (RTX 3060).

Requirements:
- cmake_minimum_required(VERSION 3.28)
- project(WhetstoneAI_Fabricate LANGUAGES CXX CUDA)
[... full CMake spec with find_package, targets, link libraries ...]

Returned generatedCode:

void generated(std::string input) {
    printf(input);
}

Returned quality:

{
  "gates": {
    "compile": { "passed": false, "reason": "code_missing_structure_or_fallback" },
    "failure_reasons": ["compile_estimate_failed", "fallback_generation_detected"],
    "overall_ready": false
  },
  "quality": {
    "compilable_estimate": false,
    "fallback_used": true,
    "warnings": ["compile_estimate_failed", "fallback_generation_detected"]
  }
}

What happened: The tool recognized it couldn't handle CMake (fallback_generation_detected, overall_ready: false) but instead of returning an error or empty result, it returned a completely unrelated C++ printf stub. The fallback detection is correct — the response format is wrong. A caller that checks overall_ready will discard it, but a caller that reads generatedCode directly will get garbage.

Expected behavior: Either return an error message explaining CMake is not a supported language in cpp mode, or return empty generatedCode with a clear unsupported_language failure reason.

---

Bug 2 (more serious): Detailed C++ spec returns generic stub, gates report success

Tool: whetstone_generate_code Input spec (verbatim):

Generate a C++17 header file: include/trajectory_writer.h

This is the TrajectoryWriter class for a CUDA ML training logger. It writes a binary
trajectory file in the locked FABR format.

Dependencies: #include "reflective_tensor.h" (provides WeightMorphism struct: alignas(16),
fields uint64_t source_node_id, float sensitivity, uint32_t logic_axiom_id),
#include <cstdint>, #include <string>, #include <vector>, #include <fstream>

Class TrajectoryWriter:
- Constructor: TrajectoryWriter(const std::string& path, uint64_t num_params)
  Opens file at path for binary write. Writes the file header immediately:
    magic:      4 bytes  "FABR"
    version:    1 byte   0x01
    num_params: uint64_t (little-endian native)
    reserved:   7 bytes  all zero
  Throws std::runtime_error if file cannot be opened.
- void write_initial_weights(const float* weights)
  Writes raw float32[num_params] immediately after the header. Must be called exactly
  once before write_step.
- void write_step(uint64_t step_idx, const float* deltas,
                  const std::vector<WeightMorphism>& morphisms)
  Writes one per-step record: [step_index, morphism_count, morphism_data,
  compressed_size, compressed_data]. Byte-shuffle + Zstd compress.
- Destructor: closes the file.
- Private members: std::ofstream file_, uint64_t num_params_
- No copy constructor or assignment operator (delete both).

Also declare a free function:
  bool verify_trajectory(const std::string& path, uint64_t expected_steps)
  [description of reader behavior]

Returned generatedCode:

#pragma once
#include <string>

namespace generated_module {

class GeneratedClass {
public:
    void run();
};

}  // namespace generated_module

Returned quality:

{
  "gates": {
    "compile": { "passed": true, "reason": "ok" },
    "failure_reasons": [],
    "overall_ready": true,
    "placeholder": { "passed": true, "reason": "ok" },
    "tests": { "passed": true, "reason": "ok" }
  },
  "quality": {
    "compilable_estimate": true,
    "fallback_used": false,
    "placeholder_count": 0,
    "todo_count": 0,
    "warnings": []
  },
  "usedSymbols": []
}

What happened: The spec described a class with a specific name (TrajectoryWriter), 4 methods with specific signatures, specific includes, a deleted copy constructor, and a free function (verify_trajectory). The generated code has:

• Wrong class name (GeneratedClass vs TrajectoryWriter)
• Wrong namespace (generated_module — not in spec)
• 1 method (run()) vs 4 specified methods
• None of the specified includes
• No free function
• No deleted copy constructor

Critically: overall_ready: true, fallback_used: false, compilable_estimate: true. The gates all passed. The code compiles, but it is completely wrong. There is zero correspondence between the spec and the output beyond "it is a class in a header file."

This is the dangerous case: a caller trusting the quality gates would accept this output.

---

Hypothesis

The code generator appears to be pattern-matching on structural keywords in the spec ("class", "header") and emitting a minimal valid skeleton, but is not:

1. Extracting method signatures from the spec text
2. Extracting field names or types
3. Using the specified class name
4. Including specified dependencies
5. Generating any method bodies or stubs

The quality gates are measuring compilability of the generated code in isolation, not correspondence to the spec. A generic class Foo { void run(); } will always pass compile/placeholder/test gates.

---

Suggested Fix Directions

For Bug 1: Add an unsupported_language quality gate failure that fires when the spec appears to describe a non-C++ artifact (CMakeLists, shell script, JSON schema, etc.) when running in --language cpp mode. Return empty generatedCode with a clear error, not a C++ stub.

For Bug 2: The spec→AST extraction pipeline is not pulling method signatures. The class name from the spec should become the AST ClassDeclaration name. Method signatures described in the spec (name, parameters, return type) should each become MethodDeclaration nodes. At minimum, a "spec fidelity" gate should fire if the generated class name does not match any name mentioned in the spec.

---

Reproducibility

• Mode: --language cpp
• Tool: whetstone_generate_code
• Both bugs reproduce on the exact spec strings above, sent as the spec parameter.
• No prior start_recording was called in this session (no LoRA data captured).
• Session started fresh from /home/bill/Documents (single PID confirmed: 3955).