diff --git a/PROGRESS.md b/PROGRESS.md index 631be5f..969a6c3 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -118,13 +118,19 @@ All 38 steps implemented and passing. Each step has a corresponding test (`step1 > `TreeSitterParser` in `Parser.h` implements full CST-to-AST conversion for all three languages > with auto-annotation (Python/Elisp → @Reclaim(Tracing), C++ → memory pattern detection). -### Phase 3c: Classical Editing Mode (Steps 50–54) — TDD STUBS ONLY -- [ ] Step 50: Text editor component — not started -- [x] Step 51: TDD test written (does not link — depends on unimplemented code) +### Phase 3c: Classical Editing Mode (Steps 50–54) — IN PROGRESS +- [x] Step 50: **IMPLEMENTED** — TextEditor component + TextASTSync (8/8 tests pass) +- [x] Step 51: **IMPLEMENTED** — Text↔AST bidirectional sync with debounce (5/5 tests pass) - [ ] Step 52: Syntax highlighting — not started -- [x] Step 53: TDD test written (does not link) +- [x] Step 53: **IMPLEMENTED** — Classical editing ops: undo/redo, find/replace, selection (6/6 tests pass) - [ ] Step 54: Emacs-style keybindings — not started +> Steps 50–53: `TextASTSync` in `editor/src/TextASTSync.h` provides bidirectional +> text↔AST sync with debounce. `TextEditor` in `editor/src/TextEditor.h` wraps text +> buffer with edit operations (insert/delete/replace), undo/redo that tracks AST state, +> find/replace, and selection. Both use TreeSitterParser for text→AST and +> PythonGenerator/CppGenerator/ElispGenerator for AST→text. 19/19 tests pass. + ### Phase 3d: Emacs Integration Complete (Steps 55–58) — TDD STUBS ONLY - [ ] Step 55: Emacs splash screen — not started - [x] Step 56: TDD test written (does not link) @@ -197,7 +203,8 @@ vcpkg's imgui 1.91.9 removed the `sdl2-binding` feature (only `sdl3-binding` exi ## Test Results (Last Verified) **Steps 1–49:** All compile and pass (49 executables in `editor/build/Release/`) -**Steps 50–75:** Either not started or TDD stubs that don't link yet +**Steps 50, 51, 53:** All compile and pass (step50: 8/8, step51: 5/5, step53: 6/6) +**Steps 52, 54–75:** Either not started or TDD stubs that don't link yet The build compiles `whetstone_editor`, `orchestrator`, and steps 1–49. Steps 45–49 link against tree-sitter core + grammar static libraries. Steps 51+ are TDD stubs that reference unimplemented @@ -212,7 +219,9 @@ code and will fail to link until their implementations exist. | `editor/src/ast/ASTNode.h` | All 33+ AST node classes, JSON serialization | | `editor/src/ast/Generator.h` | PythonGenerator, CppGenerator, ElispGenerator, all canonical annotation classes | | `editor/src/ast/Schema.h` | AST schema validation | -| `editor/src/ast/Parser.h` | TreeSitterParser (stub implementations) | +| `editor/src/ast/Parser.h` | TreeSitterParser (real tree-sitter CST-to-AST for Python/C++/Elisp) | +| `editor/src/TextASTSync.h` | Bidirectional text↔AST synchronization with debounce | +| `editor/src/TextEditor.h` | Classical text editor: edit ops, undo/redo, find/replace, selection | | `editor/src/Orchestrator.h` | Orchestrator: Emacs integration, file ops, undo/redo, agent API | | `editor/src/main.cpp` | ImGui editor shell (SDL2 + OpenGL3) | | `editor/src/orchestrator_main.cpp` | Orchestrator standalone process (JSON-RPC server) | @@ -232,10 +241,11 @@ code and will fail to link until their implementations exist. ## What's Next -Phase 3b (tree-sitter integration) is complete. Next logical work: -- **Phase 3c** (Steps 50–54): Classical text editing mode (text editor component, syntax highlighting, keybindings) -- **Phase 3e** (Steps 59–63): WebSocket agent API (Step 60 already done, remaining: WebSocket endpoint, agent protocol) +Phase 3c steps 50–53 (text editing core) are complete. Next logical work: +- **Step 52** (Phase 3c): Syntax highlighting via tree-sitter color spans +- **Step 54** (Phase 3c): Emacs-style keybindings - **Phase 3d** (Steps 55–58): Emacs integration completion (splash screen, mode-specific behavior) +- **Phase 3e** (Steps 59–63): WebSocket agent API (Step 60 already done, remaining: WebSocket endpoint, agent protocol) --- @@ -255,3 +265,4 @@ Phase 3b (tree-sitter integration) is complete. Next logical work: | 2025-latest | Claude | Fixed editor hang when launched without orchestrator pipe | | 2026-02-07 | Claude Opus 4.6 | Step 60: Implemented ASTQueryAPI (tree walk, JSON output, find by type/property/annotation). 8/8 tests pass. Added PROGRESS.md. | | 2026-02-08 | Claude Opus 4.6 | Phase 3b: Real tree-sitter integration (Steps 45–49). Replaced Parser.h stubs with real tree-sitter C bindings. Python/C++/Elisp CST-to-AST conversion, memory pattern detection, error recovery. 34/34 tests pass. | +| 2026-02-08 | Claude Opus 4.6 | Phase 3c: Steps 50–53. TextASTSync (bidirectional text↔AST sync with debounce) and TextEditor (edit ops, undo/redo with AST tracking, find/replace, selection). 19/19 tests pass. Fixed step53 find-position bug (was off-by-one). | diff --git a/editor/CMakeLists.txt b/editor/CMakeLists.txt index 0d712a1..d1cd937 100644 --- a/editor/CMakeLists.txt +++ b/editor/CMakeLists.txt @@ -234,11 +234,17 @@ add_executable(step49_test tests/step49_test.cpp) target_include_directories(step49_test PRIVATE src) target_link_libraries(step49_test PRIVATE unofficial::tree-sitter::tree-sitter tree_sitter_python tree_sitter_cpp tree_sitter_elisp) +add_executable(step50_test tests/step50_test.cpp) +target_include_directories(step50_test PRIVATE src) +target_link_libraries(step50_test PRIVATE unofficial::tree-sitter::tree-sitter tree_sitter_python tree_sitter_cpp tree_sitter_elisp) + add_executable(step51_test tests/step51_test.cpp) target_include_directories(step51_test PRIVATE src) +target_link_libraries(step51_test PRIVATE unofficial::tree-sitter::tree-sitter tree_sitter_python tree_sitter_cpp tree_sitter_elisp) add_executable(step53_test tests/step53_test.cpp) target_include_directories(step53_test PRIVATE src) +target_link_libraries(step53_test PRIVATE unofficial::tree-sitter::tree-sitter tree_sitter_python tree_sitter_cpp tree_sitter_elisp) add_executable(step56_test tests/step56_test.cpp) target_include_directories(step56_test PRIVATE src) diff --git a/editor/src/TextASTSync.h b/editor/src/TextASTSync.h new file mode 100644 index 0000000..3558eee --- /dev/null +++ b/editor/src/TextASTSync.h @@ -0,0 +1,69 @@ +#pragma once +// Step 51: Text-AST bidirectional synchronization +// +// TextASTSync bridges raw text editing and the structured AST: +// setText() → stores text, marks parse pending (debounce) +// syncNow() → parses text via TreeSitterParser → updates AST +// getText() → regenerates text from current AST via generator +// getAST() → returns current Module* +// setAST() → replaces AST directly (structured edit path) + +#include "ast/Parser.h" +#include "ast/Generator.h" +#include +#include + +class TextASTSync { +public: + void setText(const std::string& text, const std::string& language) { + text_ = text; + language_ = language; + parsePending_ = true; + } + + Module* getAST() const { + return module_.get(); + } + + void setAST(std::unique_ptr module) { + module_ = std::move(module); + parsePending_ = false; + } + + std::string getText() const { + if (!module_) return text_; + if (language_ == "python") { + PythonGenerator gen; + return gen.generate(module_.get()); + } else if (language_ == "cpp") { + CppGenerator gen; + return gen.generate(module_.get()); + } else if (language_ == "elisp") { + ElispGenerator gen; + return gen.generate(module_.get()); + } + return text_; + } + + void syncNow() { + if (!parsePending_) return; + if (language_ == "python") { + module_ = TreeSitterParser::parsePython(text_); + } else if (language_ == "cpp") { + module_ = TreeSitterParser::parseCpp(text_); + } else if (language_ == "elisp") { + module_ = TreeSitterParser::parseElisp(text_); + } + parsePending_ = false; + } + + bool isParsePending() const { return parsePending_; } + + const std::string& getLanguage() const { return language_; } + +private: + std::string text_; + std::string language_; + std::unique_ptr module_; + bool parsePending_ = false; +}; diff --git a/editor/src/TextEditor.h b/editor/src/TextEditor.h new file mode 100644 index 0000000..72d3f5f --- /dev/null +++ b/editor/src/TextEditor.h @@ -0,0 +1,124 @@ +#pragma once +// Step 53: Classical text editing with AST integration +// +// TextEditor provides a text buffer with edit operations that stay +// synchronized with the AST. Every mutation (insert, delete, replace, +// replaceAll) pushes the previous text onto an undo stack, modifies the +// text buffer, then re-parses to update the AST. Undo/redo restores +// the text and re-parses. + +#include "ast/Parser.h" +#include "ast/Generator.h" +#include +#include +#include + +class TextEditor { +public: + void setContent(const std::string& text, const std::string& language) { + text_ = text; + language_ = language; + undoStack_.clear(); + redoStack_.clear(); + reParse(); + } + + std::string getContent() const { return text_; } + + Module* getAST() const { return module_.get(); } + + // --- Edit operations --------------------------------------------------- + + void insertText(int position, const std::string& text) { + pushUndo(); + text_.insert(position, text); + reParse(); + } + + void deleteText(int position, int length) { + pushUndo(); + text_.erase(position, length); + reParse(); + } + + void replaceText(int position, int length, const std::string& replacement) { + pushUndo(); + text_.erase(position, length); + text_.insert(position, replacement); + reParse(); + } + + // --- Undo / Redo ------------------------------------------------------- + + void undo() { + if (undoStack_.empty()) return; + redoStack_.push_back(text_); + text_ = undoStack_.back(); + undoStack_.pop_back(); + reParse(); + } + + void redo() { + if (redoStack_.empty()) return; + undoStack_.push_back(text_); + text_ = redoStack_.back(); + redoStack_.pop_back(); + reParse(); + } + + bool canUndo() const { return !undoStack_.empty(); } + bool canRedo() const { return !redoStack_.empty(); } + + // --- Find / Replace ---------------------------------------------------- + + int find(const std::string& query, int startPos = 0) const { + size_t pos = text_.find(query, startPos); + return pos == std::string::npos ? -1 : static_cast(pos); + } + + int replaceAll(const std::string& query, const std::string& replacement) { + pushUndo(); + int count = 0; + size_t pos = 0; + while ((pos = text_.find(query, pos)) != std::string::npos) { + text_.erase(pos, query.length()); + text_.insert(pos, replacement); + pos += replacement.length(); + ++count; + } + if (count > 0) { + reParse(); + } else { + undoStack_.pop_back(); // no change — discard undo entry + } + return count; + } + + // --- Selection --------------------------------------------------------- + + std::string getSelection(int start, int length) const { + return text_.substr(start, length); + } + +private: + void pushUndo() { + undoStack_.push_back(text_); + redoStack_.clear(); + } + + void reParse() { + if (language_ == "python") { + module_ = TreeSitterParser::parsePython(text_); + } else if (language_ == "cpp") { + module_ = TreeSitterParser::parseCpp(text_); + } else if (language_ == "elisp") { + module_ = TreeSitterParser::parseElisp(text_); + } + } + + std::string text_; + std::string language_; + std::unique_ptr module_; + std::vector undoStack_; + std::vector redoStack_; +}; diff --git a/editor/tests/step50_test.cpp b/editor/tests/step50_test.cpp new file mode 100644 index 0000000..0e4cabc --- /dev/null +++ b/editor/tests/step50_test.cpp @@ -0,0 +1,139 @@ +// Step 50 TDD Test: Text editor component +// +// Validates the TextEditor and TextASTSync components exist and +// basic operations work: create, set content, get content, basic +// insert/delete, language selection. + +#include +#include +#include +#include "TextASTSync.h" +#include "TextEditor.h" + +static bool contains(const std::string& haystack, const std::string& needle) { + return haystack.find(needle) != std::string::npos; +} + +int main() { + int passed = 0; + int failed = 0; + + // --- Test 1: TextASTSync default state --- + { + TextASTSync sync; + assert(sync.getAST() == nullptr && "Default AST should be null"); + assert(!sync.isParsePending() && "No parse should be pending initially"); + + std::cout << "Test 1 PASS: TextASTSync default state" << std::endl; + ++passed; + } + + // --- Test 2: TextEditor set/get content --- + { + TextEditor editor; + editor.setContent("hello world", "python"); + assert(editor.getContent() == "hello world" && "Content should match"); + + std::cout << "Test 2 PASS: TextEditor set/get content" << std::endl; + ++passed; + } + + // --- Test 3: TextEditor insert --- + { + TextEditor editor; + editor.setContent("hello world", "python"); + editor.insertText(5, " beautiful"); + assert(editor.getContent() == "hello beautiful world" && "Insert should work"); + + std::cout << "Test 3 PASS: TextEditor insert" << std::endl; + ++passed; + } + + // --- Test 4: TextEditor delete --- + { + TextEditor editor; + editor.setContent("hello world", "python"); + editor.deleteText(5, 6); // delete " world" + assert(editor.getContent() == "hello" && "Delete should work"); + + std::cout << "Test 4 PASS: TextEditor delete" << std::endl; + ++passed; + } + + // --- Test 5: TextASTSync Python parse produces valid AST --- + { + TextASTSync sync; + sync.setText("def test_func(x, y):\n return x + y\n", "python"); + sync.syncNow(); + + Module* ast = sync.getAST(); + assert(ast != nullptr && "AST should exist after sync"); + assert(ast->targetLanguage == "python" && "Language should be python"); + + auto functions = ast->getChildren("functions"); + assert(functions.size() == 1 && "Should have one function"); + + auto* fn = static_cast(functions[0]); + assert(fn->name == "test_func" && "Function name should match"); + + auto params = fn->getChildren("parameters"); + assert(params.size() == 2 && "Should have two parameters"); + + std::cout << "Test 5 PASS: Python parse produces valid AST" << std::endl; + ++passed; + } + + // --- Test 6: TextEditor produces AST from content --- + { + TextEditor editor; + editor.setContent("def my_func():\n pass\n", "python"); + + Module* ast = editor.getAST(); + assert(ast != nullptr && "TextEditor should produce AST"); + + auto functions = ast->getChildren("functions"); + assert(!functions.empty() && "Should have parsed function"); + + auto* fn = static_cast(functions[0]); + assert(fn->name == "my_func" && "Function name should match"); + + std::cout << "Test 6 PASS: TextEditor produces AST from content" << std::endl; + ++passed; + } + + // --- Test 7: TextEditor undo/redo availability --- + { + TextEditor editor; + editor.setContent("test", "python"); + assert(!editor.canUndo() && "Should not be able to undo initially"); + assert(!editor.canRedo() && "Should not be able to redo initially"); + + editor.insertText(4, "!"); + assert(editor.canUndo() && "Should be able to undo after edit"); + + editor.undo(); + assert(editor.canRedo() && "Should be able to redo after undo"); + assert(editor.getContent() == "test" && "Undo should restore text"); + + std::cout << "Test 7 PASS: Undo/redo availability tracking" << std::endl; + ++passed; + } + + // --- Test 8: TextASTSync C++ parse --- + { + TextASTSync sync; + sync.setText("void hello() { return; }", "cpp"); + sync.syncNow(); + + Module* ast = sync.getAST(); + assert(ast != nullptr && "C++ AST should exist"); + assert(ast->targetLanguage == "cpp" && "Language should be cpp"); + + std::cout << "Test 8 PASS: C++ parse via TextASTSync" << std::endl; + ++passed; + } + + // --- Summary --- + std::cout << "\n=== Step 50 Results: " << passed << " passed, " << failed << " failed ===" << std::endl; + return failed > 0 ? 1 : 0; +} diff --git a/editor/tests/step51_test.cpp b/editor/tests/step51_test.cpp index dabd2d8..983e479 100644 --- a/editor/tests/step51_test.cpp +++ b/editor/tests/step51_test.cpp @@ -1,52 +1,28 @@ -// Step 51 TDD Test: Text → AST synchronization +// Step 51 TDD Test: Text -> AST synchronization // // Tests bidirectional sync between text editing and AST: -// 1. Text change → re-parse via tree-sitter → AST updates -// 2. AST change (structured edit) → regenerate text -// 3. Round-trip: text → AST → text produces semantically equivalent output +// 1. Text change -> re-parse via tree-sitter -> AST updates +// 2. AST change (structured edit) -> regenerate text +// 3. Round-trip: text -> AST -> text produces semantically equivalent output // 4. Debouncing: rapid text changes don't cause excessive re-parsing // -// Will fail until the text-AST sync layer is implemented. +// Depends on: TextASTSync, TreeSitterParser, PythonGenerator, CppGenerator #include #include #include #include -#include "ast/Parser.h" -#include "ast/Generator.h" +#include "TextASTSync.h" static bool contains(const std::string& haystack, const std::string& needle) { return haystack.find(needle) != std::string::npos; } -// Forward declaration for the sync class that doesn't exist yet -// This will need to be implemented as part of Step 51 -class TextASTSync { -public: - // Set the source text; triggers re-parse (possibly debounced) - void setText(const std::string& text, const std::string& language); - - // Get the current AST root - Module* getAST() const; - - // Mutate the AST; triggers text regeneration - void setAST(std::unique_ptr module); - - // Get the current text representation - std::string getText() const; - - // Force immediate sync (skip debounce) - void syncNow(); - - // Check if a re-parse is pending (debounced but not yet executed) - bool isParsePending() const; -}; - int main() { int passed = 0; int failed = 0; - // --- Test 1: Set text → AST updates --- + // --- Test 1: Set text -> AST updates --- { TextASTSync sync; sync.setText("def greet(name):\n return name\n", "python"); @@ -61,11 +37,11 @@ int main() { auto* fn = static_cast(functions[0]); assert(fn->name == "greet" && "Function name should be 'greet'"); - std::cout << "Test 1 PASS: setText → AST has correct function" << std::endl; + std::cout << "Test 1 PASS: setText -> AST has correct function" << std::endl; ++passed; } - // --- Test 2: AST change → text updates --- + // --- Test 2: AST change -> text updates --- { TextASTSync sync; sync.setText("def f(x):\n return x\n", "python"); @@ -83,11 +59,11 @@ int main() { assert(contains(newText, "compute") && "Text should reflect renamed function"); assert(!contains(newText, "def f(") && "Old function name should be gone"); - std::cout << "Test 2 PASS: AST change → text updates" << std::endl; + std::cout << "Test 2 PASS: AST change -> text updates" << std::endl; ++passed; } - // --- Test 3: Round-trip text → AST → text --- + // --- Test 3: Round-trip text -> AST -> text --- { std::string original = "def add(a, b):\n return a + b\n"; TextASTSync sync; @@ -108,11 +84,11 @@ int main() { ++passed; } - // --- Test 4: Debounce — rapid changes don't cause immediate re-parse --- + // --- Test 4: Debounce --- rapid changes don't cause immediate re-parse --- { TextASTSync sync; sync.setText("def a():\n pass\n", "python"); - // Don't call syncNow — simulate rapid edits + // Don't call syncNow --- simulate rapid edits sync.setText("def ab():\n pass\n", "python"); sync.setText("def abc():\n pass\n", "python"); @@ -133,7 +109,7 @@ int main() { ++passed; } - // --- Test 5: C++ text → AST sync --- + // --- Test 5: C++ text -> AST sync --- { TextASTSync sync; sync.setText("int f(int x) { return x + 1; }", "cpp"); @@ -146,7 +122,7 @@ int main() { auto functions = ast->getChildren("functions"); assert(!functions.empty() && "Should parse C++ function"); - std::cout << "Test 5 PASS: C++ text → AST sync works" << std::endl; + std::cout << "Test 5 PASS: C++ text -> AST sync works" << std::endl; ++passed; } diff --git a/editor/tests/step53_test.cpp b/editor/tests/step53_test.cpp index 9fd1669..c365b66 100644 --- a/editor/tests/step53_test.cpp +++ b/editor/tests/step53_test.cpp @@ -4,54 +4,21 @@ // 1. Undo in text mode undoes both text and AST changes // 2. Redo restores both text and AST // 3. Find text in the text buffer -// 4. Replace text → AST updates accordingly +// 4. Replace text -> AST updates accordingly // 5. Copy/paste operations work in text mode // 6. Text undo/redo integrates with orchestrator's operation journal // -// Will fail until classical editing operations are implemented. +// Depends on: TextEditor, TreeSitterParser, PythonGenerator #include #include #include -#include "ast/ASTNode.h" -#include "ast/Module.h" -#include "ast/Function.h" +#include "TextEditor.h" static bool contains(const std::string& haystack, const std::string& needle) { return haystack.find(needle) != std::string::npos; } -// Forward declaration — TextEditor class that wraps text ops + AST sync -class TextEditor { -public: - // Set initial content - void setContent(const std::string& text, const std::string& language); - - // Get current text - std::string getContent() const; - - // Get current AST - Module* getAST() const; - - // Edit operations - void insertText(int position, const std::string& text); - void deleteText(int position, int length); - void replaceText(int position, int length, const std::string& replacement); - - // Undo/redo (must also undo/redo AST changes) - void undo(); - void redo(); - bool canUndo() const; - bool canRedo() const; - - // Find/replace - int find(const std::string& query, int startPos = 0) const; - int replaceAll(const std::string& query, const std::string& replacement); - - // Clipboard - std::string getSelection(int start, int length) const; -}; - int main() { int passed = 0; int failed = 0; @@ -112,7 +79,7 @@ int main() { editor.replaceText(4, 5, "world"); editor.undo(); - assert(!editor.canRedo() == false && "Should be able to redo after undo"); + assert(editor.canRedo() && "Should be able to redo after undo"); editor.redo(); std::string text = editor.getContent(); @@ -134,7 +101,9 @@ int main() { int pos = editor.find("return"); assert(pos >= 0 && "Should find 'return' in text"); - assert(pos == 18 && "Position of 'return' should be at column after indent"); + // "def hello():\n return 42\n" + // 0123456789012 34567... -> 'r' of "return" is at index 17 + assert(pos == 17 && "Position of 'return' should be at index 17"); int notFound = editor.find("nonexistent"); assert(notFound == -1 && "Should return -1 for not found");