Step 227: Open-source model tool definitions

PROGRESS.md

@@ -746,3 +746,4 @@ vcpkg's imgui 1.91.9 removed the `sdl2-binding` feature (only `sdl3-binding` exi
 | 2026-02-10 | Claude Opus 4.6 | Sprint 7 Phase 7a (Steps 202–206): API docs (AGENT_API.md, 23 methods), JSON schemas (20 files), exposed ContextAPI/BatchMutationAPI/Pipeline via RPC, permission policy updates. 51/51 tests pass. |
 | 2026-02-10 | Claude Opus 4.6 | Sprint 7 Phase 7b (Steps 207–213): MCPServer.h (10 tools, 5 resources, 4 prompts, JSON-RPC 2.0 initialize handshake), MCPBridge.h (stdio transport). 90/90 tests pass. |
 | 2026-02-10 | Claude Opus 4.6 | Sprint 7 Phase 7c (Steps 214–219): TraceGenerator.h (6 scenario templates, 9-sample code corpus, batch engine), TraceExporter.h (Anthropic/OpenAI/JSONL/Markdown export, filtering, statistics). 294/294 tests pass. |
+| 2026-02-10 | Codex | Step 227: Open-source model tool definitions (ReAct-style prompt, XML tool call format, simplified schemas, adapter rules). 1/1 tests pass. |

editor/CMakeLists.txt

@@ -1164,6 +1164,8 @@ target_include_directories(step225_test PRIVATE src)
 
 add_executable(step226_test tests/step226_test.cpp)
 target_include_directories(step226_test PRIVATE src)
+add_executable(step227_test tests/step227_test.cpp)
+target_include_directories(step227_test PRIVATE src)
 add_executable(step213_test tests/step213_test.cpp)
 target_include_directories(step213_test PRIVATE src)
 target_link_libraries(step213_test PRIVATE nlohmann_json::nlohmann_json)

editor/tests/step227_test.cpp

@@ -0,0 +1,41 @@
+// Step 227: Open-source model tool definitions.
+
+#include <cassert>
+#include <fstream>
+#include <string>
+
+static std::string readFile(const std::string& path) {
+    std::ifstream f(path);
+    if (!f.is_open()) return {};
+    return std::string((std::istreambuf_iterator<char>(f)),
+                       std::istreambuf_iterator<char>());
+}
+
+static void assertContains(const std::string& text, const std::string& needle) {
+    assert(text.find(needle) != std::string::npos);
+}
+
+int main() {
+    const std::string tools = readFile("../tools/generic/tools.json");
+    const std::string prompt = readFile("../tools/generic/system_prompt.txt");
+    const std::string adapter = readFile("../tools/generic/adapter.json");
+
+    assertContains(tools, "whetstone_get_ast");
+    assertContains(tools, "whetstone_mutate");
+    assertContains(tools, "whetstone_batch_mutate");
+    assertContains(tools, "whetstone_get_scope");
+    assertContains(tools, "whetstone_get_call_hierarchy");
+    assertContains(tools, "whetstone_suggest_annotations");
+    assertContains(tools, "whetstone_apply_annotation");
+    assertContains(tools, "whetstone_generate_code");
+    assertContains(tools, "whetstone_run_pipeline");
+    assertContains(tools, "whetstone_project_language");
+
+    assertContains(prompt, "ReAct");
+    assertContains(prompt, "<tool_call");
+    assertContains(adapter, "tool_call_tag");
+    assertContains(adapter, "tool_result_tag");
+
+    printf("step227_test: all assertions passed\n");
+    return 0;
+}

sprint7_plan.md

@@ -367,7 +367,7 @@ Optimized tool definitions and prompts for specific LLM families.
   - Parallel tool calling hints for independent operations
   *New:* `tools/openai/` directory with function definitions + system prompt
 
-- [ ] **Step 227: Open-source model tool definitions**
+- [x] **Step 227: Open-source model tool definitions**
   Generic tool definitions for open-source models (Llama, Mistral, etc.):
   - ReAct-style prompting with tool descriptions in system prompt
   - XML-tagged tool call format (for models without native tool calling)

tools/generic/adapter.json

@@ -0,0 +1,36 @@
+{
+  "version": "1.0",
+  "call_format": {
+    "tool_call_tag": "tool_call",
+    "name_attribute": "name",
+    "payload": "json",
+    "allow_whitespace": true,
+    "max_calls_per_message": 1
+  },
+  "result_format": {
+    "tool_result_tag": "tool_result",
+    "name_attribute": "name",
+    "payload": "json"
+  },
+  "parsing": {
+    "strip_markdown_fences": true,
+    "accept_action_prefix": "Action:",
+    "accept_thought_prefix": "Thought:",
+    "error_on_multiple_calls": true
+  },
+  "defaults": {
+    "whetstone_generate_code": { "preferImports": true }
+  },
+  "mapping": {
+    "whetstone_get_ast": "whetstone_get_ast",
+    "whetstone_mutate": "whetstone_mutate",
+    "whetstone_batch_mutate": "whetstone_batch_mutate",
+    "whetstone_get_scope": "whetstone_get_scope",
+    "whetstone_get_call_hierarchy": "whetstone_get_call_hierarchy",
+    "whetstone_suggest_annotations": "whetstone_suggest_annotations",
+    "whetstone_apply_annotation": "whetstone_apply_annotation",
+    "whetstone_generate_code": "whetstone_generate_code",
+    "whetstone_run_pipeline": "whetstone_run_pipeline",
+    "whetstone_project_language": "whetstone_project_language"
+  }
+}

tools/generic/system_prompt.txt

@@ -0,0 +1,22 @@
+You are a Whetstone coding assistant for open-source LLMs. Use ReAct-style steps: think, then call tools.
+
+Tool call format (XML):
+Thought: brief reasoning.
+Action: <tool_call name="TOOL_NAME">{JSON_ARGS}</tool_call>
+
+Rules:
+- Use exactly one tool call per Action.
+- Emit valid JSON inside the tool call tag.
+- If no tool is needed, respond with a direct answer.
+
+Available tools:
+- whetstone_get_ast: Return the current AST.
+- whetstone_mutate: Apply a single AST mutation.
+- whetstone_batch_mutate: Apply multiple mutations atomically.
+- whetstone_get_scope: List symbols in scope.
+- whetstone_get_call_hierarchy: Get call hierarchy for a function.
+- whetstone_suggest_annotations: Suggest memory annotations.
+- whetstone_apply_annotation: Apply memory annotations.
+- whetstone_generate_code: Generate code from a spec.
+- whetstone_run_pipeline: Run the full pipeline (source -> target).
+- whetstone_project_language: Project AST to a target language.

tools/generic/tools.json

@@ -0,0 +1,106 @@
+{
+  "version": "1.0",
+  "provider": "generic",
+  "call_format": {
+    "type": "xml",
+    "tag": "tool_call",
+    "name_attribute": "name",
+    "payload": "json",
+    "example": "<tool_call name=\"whetstone_get_ast\">{}</tool_call>"
+  },
+  "tools": [
+    {
+      "name": "whetstone_get_ast",
+      "description": "Return the current AST for the active buffer. Use before mutations.",
+      "args": [],
+      "defaults": {}
+    },
+    {
+      "name": "whetstone_mutate",
+      "description": "Apply a single AST mutation. Use setProperty, updateNode, deleteNode, or insertNode.",
+      "args": [
+        { "name": "type", "type": "string", "required": true, "enum": ["setProperty", "updateNode", "deleteNode", "insertNode"] },
+        { "name": "nodeId", "type": "string", "required": false, "notes": "Required for setProperty, updateNode, deleteNode." },
+        { "name": "property", "type": "string", "required": false },
+        { "name": "value", "type": "string", "required": false },
+        { "name": "parentId", "type": "string", "required": false },
+        { "name": "role", "type": "string", "required": false },
+        { "name": "node", "type": "object", "required": false }
+      ],
+      "defaults": {}
+    },
+    {
+      "name": "whetstone_batch_mutate",
+      "description": "Apply multiple mutations atomically.",
+      "args": [
+        { "name": "mutations", "type": "array", "required": true }
+      ],
+      "defaults": {}
+    },
+    {
+      "name": "whetstone_get_scope",
+      "description": "List symbols in scope for a node.",
+      "args": [
+        { "name": "nodeId", "type": "string", "required": true }
+      ],
+      "defaults": {}
+    },
+    {
+      "name": "whetstone_get_call_hierarchy",
+      "description": "Get call hierarchy for a function.",
+      "args": [
+        { "name": "functionId", "type": "string", "required": true }
+      ],
+      "defaults": {}
+    },
+    {
+      "name": "whetstone_suggest_annotations",
+      "description": "Suggest memory annotations for a node or line/col location.",
+      "args": [
+        { "name": "nodeId", "type": "string", "required": false },
+        { "name": "line", "type": "integer", "required": false },
+        { "name": "col", "type": "integer", "required": false }
+      ],
+      "defaults": {}
+    },
+    {
+      "name": "whetstone_apply_annotation",
+      "description": "Apply a memory annotation suggestion.",
+      "args": [
+        { "name": "nodeId", "type": "string", "required": true },
+        { "name": "annotationType", "type": "string", "required": true },
+        { "name": "strategy", "type": "string", "required": true },
+        { "name": "reason", "type": "string", "required": false },
+        { "name": "confidence", "type": "number", "required": false }
+      ],
+      "defaults": {}
+    },
+    {
+      "name": "whetstone_generate_code",
+      "description": "Generate code from a natural language spec.",
+      "args": [
+        { "name": "spec", "type": "string", "required": true },
+        { "name": "preferImports", "type": "boolean", "required": false }
+      ],
+      "defaults": { "preferImports": true }
+    },
+    {
+      "name": "whetstone_run_pipeline",
+      "description": "Run the full pipeline from source to target language.",
+      "args": [
+        { "name": "source", "type": "string", "required": true },
+        { "name": "sourceLanguage", "type": "string", "required": true },
+        { "name": "targetLanguage", "type": "string", "required": true }
+      ],
+      "defaults": {}
+    },
+    {
+      "name": "whetstone_project_language",
+      "description": "Project the current AST to a target language.",
+      "args": [
+        { "name": "targetLanguage", "type": "string", "required": true }
+      ],
+      "defaults": {}
+    }
+  ]
+}