Phase 3c complete: syntax highlighting and configurable keybindings (Steps 52, 54)

Step 52: SyntaxHighlighter walks tree-sitter CST to produce colored
token spans (keyword, string, comment, number, function, parameter,
type, operator, etc.) for Python, C++, and Elisp.

Step 54: KeybindingManager with swappable profiles — VSCode (default),
JetBrains, and Emacs. Each profile maps 25+ actions (file, edit,
search, nav, view, code, build) to key combos. Supports custom
overrides and reverse lookup.

Design direction: editor targets VSCode/JetBrains look-and-feel.
Keybinding profile selectable at startup. VSCode is default.

Phase 3c totals: 37/37 tests pass across 5 steps (50-54).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Bill
2026-02-08 16:12:10 -07:00
parent 742c33d813
commit 5190e12b89
6 changed files with 1090 additions and 13 deletions

View File

@@ -118,18 +118,21 @@ All 38 steps implemented and passing. Each step has a corresponding test (`step1
> `TreeSitterParser` in `Parser.h` implements full CST-to-AST conversion for all three languages
> with auto-annotation (Python/Elisp → @Reclaim(Tracing), C++ → memory pattern detection).
### Phase 3c: Classical Editing Mode (Steps 5054) — IN PROGRESS
### Phase 3c: Classical Editing Mode (Steps 5054) — COMPLETE
- [x] Step 50: **IMPLEMENTED** — TextEditor component + TextASTSync (8/8 tests pass)
- [x] Step 51: **IMPLEMENTED** — Text↔AST bidirectional sync with debounce (5/5 tests pass)
- [ ] Step 52: Syntax highlighting — not started
- [x] Step 52: **IMPLEMENTED** — Syntax highlighting via tree-sitter CST walk (8/8 tests pass)
- [x] Step 53: **IMPLEMENTED** — Classical editing ops: undo/redo, find/replace, selection (6/6 tests pass)
- [ ] Step 54: Emacs-style keybindings — not started
- [x] Step 54: **IMPLEMENTED** — Configurable keybinding profiles: VSCode (default), JetBrains, Emacs (10/10 tests pass)
> Steps 5053: `TextASTSync` in `editor/src/TextASTSync.h` provides bidirectional
> textAST sync with debounce. `TextEditor` in `editor/src/TextEditor.h` wraps text
> buffer with edit operations (insert/delete/replace), undo/redo that tracks AST state,
> find/replace, and selection. Both use TreeSitterParser for text→AST and
> PythonGenerator/CppGenerator/ElispGenerator for AST→text. 19/19 tests pass.
> Phase 3c complete (37/37 tests pass). Backend components:
> - `TextASTSync.h`: bidirectional text↔AST sync with debounce
> - `TextEditor.h`: edit ops, undo/redo with AST tracking, find/replace, selection
> - `SyntaxHighlighter.h`: tree-sitter CST walk → colored spans (keyword, string, comment, etc.)
> - `KeybindingManager.h`: swappable profiles (VSCode/JetBrains/Emacs), custom overrides
>
> **Design direction:** Editor UI targets VSCode/JetBrains look-and-feel (not Emacs).
> Keybinding profile is selectable at startup or via settings. VSCode is default.
### Phase 3d: Emacs Integration Complete (Steps 5558) — TDD STUBS ONLY
- [ ] Step 55: Emacs splash screen — not started
@@ -203,8 +206,8 @@ vcpkg's imgui 1.91.9 removed the `sdl2-binding` feature (only `sdl3-binding` exi
## Test Results (Last Verified)
**Steps 149:** All compile and pass (49 executables in `editor/build/Release/`)
**Steps 50, 51, 53:** All compile and pass (step50: 8/8, step51: 5/5, step53: 6/6)
**Steps 52, 5475:** Either not started or TDD stubs that don't link yet
**Steps 5054:** All compile and pass (step50: 8/8, step51: 5/5, step52: 8/8, step53: 6/6, step54: 10/10)
**Steps 5575:** Either not started or TDD stubs that don't link yet
The build compiles `whetstone_editor`, `orchestrator`, and steps 149. Steps 4549 link against
tree-sitter core + grammar static libraries. Steps 51+ are TDD stubs that reference unimplemented
@@ -222,6 +225,8 @@ code and will fail to link until their implementations exist.
| `editor/src/ast/Parser.h` | TreeSitterParser (real tree-sitter CST-to-AST for Python/C++/Elisp) |
| `editor/src/TextASTSync.h` | Bidirectional text↔AST synchronization with debounce |
| `editor/src/TextEditor.h` | Classical text editor: edit ops, undo/redo, find/replace, selection |
| `editor/src/SyntaxHighlighter.h` | Tree-sitter CST walk → colored token spans (Python/C++/Elisp) |
| `editor/src/KeybindingManager.h` | Configurable keybinding profiles (VSCode/JetBrains/Emacs) |
| `editor/src/Orchestrator.h` | Orchestrator: Emacs integration, file ops, undo/redo, agent API |
| `editor/src/main.cpp` | ImGui editor shell (SDL2 + OpenGL3) |
| `editor/src/orchestrator_main.cpp` | Orchestrator standalone process (JSON-RPC server) |
@@ -241,11 +246,12 @@ code and will fail to link until their implementations exist.
## What's Next
Phase 3c steps 5053 (text editing core) are complete. Next logical work:
- **Step 52** (Phase 3c): Syntax highlighting via tree-sitter color spans
- **Step 54** (Phase 3c): Emacs-style keybindings
Phase 3c (Classical Editing Mode) is complete. All backend components exist:
text editing, AST sync, syntax highlighting, and configurable keybindings.
Next logical work:
- **Phase 3d** (Steps 5558): Emacs integration completion (splash screen, mode-specific behavior)
- **Phase 3e** (Steps 5963): WebSocket agent API (Step 60 already done, remaining: WebSocket endpoint, agent protocol)
- **GUI wiring**: Connect TextEditor, SyntaxHighlighter, KeybindingManager into main.cpp ImGui shell
---
@@ -266,3 +272,4 @@ Phase 3c steps 5053 (text editing core) are complete. Next logical work:
| 2026-02-07 | Claude Opus 4.6 | Step 60: Implemented ASTQueryAPI (tree walk, JSON output, find by type/property/annotation). 8/8 tests pass. Added PROGRESS.md. |
| 2026-02-08 | Claude Opus 4.6 | Phase 3b: Real tree-sitter integration (Steps 4549). Replaced Parser.h stubs with real tree-sitter C bindings. Python/C++/Elisp CST-to-AST conversion, memory pattern detection, error recovery. 34/34 tests pass. |
| 2026-02-08 | Claude Opus 4.6 | Phase 3c: Steps 5053. TextASTSync (bidirectional text↔AST sync with debounce) and TextEditor (edit ops, undo/redo with AST tracking, find/replace, selection). 19/19 tests pass. Fixed step53 find-position bug (was off-by-one). |
| 2026-02-08 | Claude Opus 4.6 | Phase 3c complete: Steps 52+54. SyntaxHighlighter (tree-sitter CST→color spans for Python/C++/Elisp). KeybindingManager (VSCode/JetBrains/Emacs profiles, default VSCode). 37/37 total Phase 3c tests pass. Design pivot: editor targets VSCode/JetBrains look, not Emacs. |

View File

@@ -242,10 +242,17 @@ add_executable(step51_test tests/step51_test.cpp)
target_include_directories(step51_test PRIVATE src)
target_link_libraries(step51_test PRIVATE unofficial::tree-sitter::tree-sitter tree_sitter_python tree_sitter_cpp tree_sitter_elisp)
add_executable(step52_test tests/step52_test.cpp)
target_include_directories(step52_test PRIVATE src)
target_link_libraries(step52_test PRIVATE unofficial::tree-sitter::tree-sitter tree_sitter_python tree_sitter_cpp tree_sitter_elisp)
add_executable(step53_test tests/step53_test.cpp)
target_include_directories(step53_test PRIVATE src)
target_link_libraries(step53_test PRIVATE unofficial::tree-sitter::tree-sitter tree_sitter_python tree_sitter_cpp tree_sitter_elisp)
add_executable(step54_test tests/step54_test.cpp)
target_include_directories(step54_test PRIVATE src)
add_executable(step56_test tests/step56_test.cpp)
target_include_directories(step56_test PRIVATE src)

View File

@@ -0,0 +1,276 @@
#pragma once
// Step 54: Configurable keybinding profiles
//
// KeybindingManager provides keybinding lookup with swappable profiles.
// Profiles: VSCode (default), JetBrains, Emacs.
// Each profile maps action names to key combinations.
// The editor queries this at runtime to dispatch key events.
#include <string>
#include <map>
#include <vector>
// Modifier flags (bitmask)
enum KeyMod : int {
MOD_NONE = 0,
MOD_CTRL = 1 << 0,
MOD_SHIFT = 1 << 1,
MOD_ALT = 1 << 2,
MOD_SUPER = 1 << 3 // Cmd on Mac, Win on Windows
};
struct KeyCombo {
int key; // virtual key or character code (e.g. 'S', 'Z', 'F')
int modifiers; // bitmask of KeyMod
bool operator==(const KeyCombo& other) const {
return key == other.key && modifiers == other.modifiers;
}
std::string toString() const {
std::string result;
if (modifiers & MOD_CTRL) result += "Ctrl+";
if (modifiers & MOD_SHIFT) result += "Shift+";
if (modifiers & MOD_ALT) result += "Alt+";
if (modifiers & MOD_SUPER) result += "Super+";
if (key >= 'A' && key <= 'Z') {
result += (char)key;
} else if (key >= '0' && key <= '9') {
result += (char)key;
} else {
result += "[" + std::to_string(key) + "]";
}
return result;
}
};
enum class KeybindingProfile {
VSCode,
JetBrains,
Emacs
};
class KeybindingManager {
public:
KeybindingManager() {
setProfile(KeybindingProfile::VSCode);
}
void setProfile(KeybindingProfile profile) {
profile_ = profile;
bindings_.clear();
switch (profile) {
case KeybindingProfile::VSCode: loadVSCode(); break;
case KeybindingProfile::JetBrains: loadJetBrains(); break;
case KeybindingProfile::Emacs: loadEmacs(); break;
}
}
KeybindingProfile getProfile() const { return profile_; }
static const char* profileName(KeybindingProfile p) {
switch (p) {
case KeybindingProfile::VSCode: return "VSCode";
case KeybindingProfile::JetBrains: return "JetBrains";
case KeybindingProfile::Emacs: return "Emacs";
}
return "Unknown";
}
static KeybindingProfile profileFromName(const std::string& name) {
if (name == "VSCode" || name == "vscode") return KeybindingProfile::VSCode;
if (name == "JetBrains" || name == "jetbrains") return KeybindingProfile::JetBrains;
if (name == "Emacs" || name == "emacs") return KeybindingProfile::Emacs;
return KeybindingProfile::VSCode; // default
}
// Look up the key combo for an action (returns empty combo if not bound)
KeyCombo getBinding(const std::string& action) const {
auto it = bindings_.find(action);
if (it != bindings_.end()) return it->second;
return {0, MOD_NONE};
}
// Look up the action for a key combo (returns "" if not bound)
std::string getAction(const KeyCombo& combo) const {
for (const auto& [action, binding] : bindings_) {
if (binding == combo) return action;
}
return "";
}
// Override a single binding
void setBinding(const std::string& action, KeyCombo combo) {
bindings_[action] = combo;
}
// List all actions with their bindings
std::vector<std::string> listActions() const {
std::vector<std::string> actions;
for (const auto& [action, _] : bindings_) {
actions.push_back(action);
}
return actions;
}
// Available profiles
static std::vector<KeybindingProfile> availableProfiles() {
return {KeybindingProfile::VSCode, KeybindingProfile::JetBrains,
KeybindingProfile::Emacs};
}
private:
void bind(const std::string& action, int key, int mods) {
bindings_[action] = {key, mods};
}
// --- VSCode profile (default) ---
void loadVSCode() {
// File operations
bind("file.save", 'S', MOD_CTRL);
bind("file.saveAs", 'S', MOD_CTRL | MOD_SHIFT);
bind("file.open", 'O', MOD_CTRL);
bind("file.new", 'N', MOD_CTRL);
bind("file.close", 'W', MOD_CTRL);
// Edit operations
bind("edit.undo", 'Z', MOD_CTRL);
bind("edit.redo", 'Y', MOD_CTRL);
bind("edit.cut", 'X', MOD_CTRL);
bind("edit.copy", 'C', MOD_CTRL);
bind("edit.paste", 'V', MOD_CTRL);
bind("edit.selectAll", 'A', MOD_CTRL);
bind("edit.delete", 127, MOD_NONE); // Delete key
bind("edit.duplicateLine", 'D', MOD_CTRL | MOD_SHIFT);
// Search
bind("search.find", 'F', MOD_CTRL);
bind("search.replace", 'H', MOD_CTRL);
bind("search.findNext", 'G', MOD_CTRL);
bind("search.findPrev", 'G', MOD_CTRL | MOD_SHIFT);
bind("search.findInFiles", 'F', MOD_CTRL | MOD_SHIFT);
// Navigation
bind("nav.goToLine", 'G', MOD_CTRL);
bind("nav.goToFile", 'P', MOD_CTRL);
bind("nav.goToSymbol", 'O', MOD_CTRL | MOD_SHIFT);
// View
bind("view.toggleSidebar", 'B', MOD_CTRL);
bind("view.toggleTerminal", '`', MOD_CTRL);
bind("view.zoomIn", '=', MOD_CTRL);
bind("view.zoomOut", '-', MOD_CTRL);
bind("view.commandPalette", 'P', MOD_CTRL | MOD_SHIFT);
// Code
bind("code.comment", '/', MOD_CTRL);
bind("code.format", 'F', MOD_CTRL | MOD_SHIFT | MOD_ALT);
bind("code.goToDefinition", 0x0D, MOD_CTRL); // Ctrl+Enter or F12
bind("code.rename", 'R', MOD_CTRL | MOD_SHIFT);
// Build
bind("build.run", 'B', MOD_CTRL | MOD_SHIFT);
bind("build.build", 'B', MOD_CTRL);
}
// --- JetBrains profile ---
void loadJetBrains() {
// File operations
bind("file.save", 'S', MOD_CTRL);
bind("file.saveAs", 'S', MOD_CTRL | MOD_SHIFT);
bind("file.open", 'O', MOD_CTRL);
bind("file.new", 'N', MOD_CTRL | MOD_ALT);
bind("file.close", 'W', MOD_CTRL);
// Edit operations
bind("edit.undo", 'Z', MOD_CTRL);
bind("edit.redo", 'Z', MOD_CTRL | MOD_SHIFT);
bind("edit.cut", 'X', MOD_CTRL);
bind("edit.copy", 'C', MOD_CTRL);
bind("edit.paste", 'V', MOD_CTRL);
bind("edit.selectAll", 'A', MOD_CTRL);
bind("edit.delete", 127, MOD_NONE);
bind("edit.duplicateLine", 'D', MOD_CTRL);
// Search
bind("search.find", 'F', MOD_CTRL);
bind("search.replace", 'R', MOD_CTRL);
bind("search.findNext", 'L', MOD_CTRL);
bind("search.findPrev", 'L', MOD_CTRL | MOD_SHIFT);
bind("search.findInFiles", 'F', MOD_CTRL | MOD_SHIFT);
// Navigation
bind("nav.goToLine", 'G', MOD_CTRL);
bind("nav.goToFile", 'N', MOD_CTRL | MOD_SHIFT);
bind("nav.goToSymbol", 'O', MOD_CTRL | MOD_ALT | MOD_SHIFT);
// View
bind("view.toggleSidebar", '1', MOD_ALT);
bind("view.toggleTerminal", '4', MOD_ALT);
bind("view.zoomIn", '=', MOD_CTRL);
bind("view.zoomOut", '-', MOD_CTRL);
bind("view.commandPalette", 'A', MOD_CTRL | MOD_SHIFT);
// Code
bind("code.comment", '/', MOD_CTRL);
bind("code.format", 'L', MOD_CTRL | MOD_ALT);
bind("code.goToDefinition", 'B', MOD_CTRL);
bind("code.rename", 'R', MOD_SHIFT | MOD_ALT);
// Build
bind("build.run", 'R', MOD_CTRL | MOD_SHIFT);
bind("build.build", 'B', MOD_CTRL | MOD_SHIFT);
}
// --- Emacs profile ---
void loadEmacs() {
// File operations
bind("file.save", 'S', MOD_CTRL);
bind("file.saveAs", 'S', MOD_CTRL | MOD_SHIFT);
bind("file.open", 'O', MOD_CTRL);
bind("file.new", 'N', MOD_CTRL);
bind("file.close", 'W', MOD_CTRL);
// Edit operations
bind("edit.undo", 'Z', MOD_CTRL);
bind("edit.redo", 'Z', MOD_CTRL | MOD_SHIFT);
bind("edit.cut", 'W', MOD_CTRL);
bind("edit.copy", 'W', MOD_ALT);
bind("edit.paste", 'Y', MOD_CTRL);
bind("edit.selectAll", 'A', MOD_CTRL);
bind("edit.delete", 'D', MOD_CTRL);
bind("edit.duplicateLine", 'D', MOD_CTRL | MOD_SHIFT);
// Search
bind("search.find", 'S', MOD_CTRL);
bind("search.replace", 'R', MOD_ALT);
bind("search.findNext", 'S', MOD_CTRL);
bind("search.findPrev", 'R', MOD_CTRL);
bind("search.findInFiles", 'F', MOD_CTRL | MOD_SHIFT);
// Navigation
bind("nav.goToLine", 'G', MOD_ALT);
bind("nav.goToFile", 'F', MOD_CTRL);
bind("nav.goToSymbol", 'O', MOD_CTRL | MOD_SHIFT);
// View
bind("view.toggleSidebar", 'B', MOD_CTRL);
bind("view.toggleTerminal", '`', MOD_CTRL);
bind("view.zoomIn", '=', MOD_CTRL);
bind("view.zoomOut", '-', MOD_CTRL);
bind("view.commandPalette", 'X', MOD_ALT);
// Code
bind("code.comment", ';', MOD_ALT);
bind("code.format", 'F', MOD_CTRL | MOD_ALT);
bind("code.goToDefinition", '.', MOD_ALT);
bind("code.rename", 'R', MOD_CTRL | MOD_SHIFT);
// Build
bind("build.run", 'C', MOD_CTRL | MOD_SHIFT);
bind("build.build", 'B', MOD_CTRL | MOD_SHIFT);
}
KeybindingProfile profile_;
std::map<std::string, KeyCombo> bindings_;
};

View File

@@ -0,0 +1,444 @@
#pragma once
// Step 52: Syntax highlighting via tree-sitter
//
// SyntaxHighlighter parses source text with tree-sitter and walks the CST
// to produce a list of colored spans. Each span has a byte range and a
// token category (keyword, string, comment, number, identifier, operator,
// type, punctuation, function, parameter).
//
// The highlight data is language-agnostic at the output level — consumers
// map categories to colors via a theme.
#include <string>
#include <vector>
#include <cstring>
#include <tree_sitter/api.h>
extern "C" {
const TSLanguage* tree_sitter_python();
const TSLanguage* tree_sitter_cpp();
const TSLanguage* tree_sitter_elisp();
}
enum class TokenCategory {
Plain,
Keyword,
String,
Comment,
Number,
Identifier,
Operator,
Type,
Punctuation,
Function,
Parameter,
Builtin
};
struct HighlightSpan {
uint32_t start; // byte offset
uint32_t end; // byte offset (exclusive)
TokenCategory category;
};
class SyntaxHighlighter {
public:
static std::vector<HighlightSpan> highlight(const std::string& source,
const std::string& language) {
std::vector<HighlightSpan> spans;
if (source.empty()) return spans;
TSParser* parser = ts_parser_new();
const TSLanguage* lang = nullptr;
if (language == "python") lang = tree_sitter_python();
else if (language == "cpp") lang = tree_sitter_cpp();
else if (language == "elisp") lang = tree_sitter_elisp();
if (!lang) {
ts_parser_delete(parser);
return spans;
}
ts_parser_set_language(parser, lang);
TSTree* tree = ts_parser_parse_string(parser, nullptr,
source.c_str(),
(uint32_t)source.size());
TSNode root = ts_tree_root_node(tree);
if (language == "python") walkPython(root, source, spans);
else if (language == "cpp") walkCpp(root, source, spans);
else if (language == "elisp") walkElisp(root, source, spans);
ts_tree_delete(tree);
ts_parser_delete(parser);
// Sort by start position
std::sort(spans.begin(), spans.end(),
[](const HighlightSpan& a, const HighlightSpan& b) {
return a.start < b.start;
});
return spans;
}
static const char* categoryName(TokenCategory cat) {
switch (cat) {
case TokenCategory::Plain: return "plain";
case TokenCategory::Keyword: return "keyword";
case TokenCategory::String: return "string";
case TokenCategory::Comment: return "comment";
case TokenCategory::Number: return "number";
case TokenCategory::Identifier: return "identifier";
case TokenCategory::Operator: return "operator";
case TokenCategory::Type: return "type";
case TokenCategory::Punctuation: return "punctuation";
case TokenCategory::Function: return "function";
case TokenCategory::Parameter: return "parameter";
case TokenCategory::Builtin: return "builtin";
}
return "plain";
}
private:
static std::string nodeType(TSNode node) {
return ts_node_type(node);
}
static std::string nodeText(TSNode node, const std::string& source) {
uint32_t s = ts_node_start_byte(node);
uint32_t e = ts_node_end_byte(node);
if (s >= source.size() || e > source.size()) return "";
return source.substr(s, e - s);
}
static void addSpan(std::vector<HighlightSpan>& spans, TSNode node,
TokenCategory cat) {
uint32_t s = ts_node_start_byte(node);
uint32_t e = ts_node_end_byte(node);
if (s < e) {
spans.push_back({s, e, cat});
}
}
// --- Python --------------------------------------------------------
static bool isPythonKeyword(const std::string& text) {
static const char* keywords[] = {
"def", "class", "if", "elif", "else", "for", "while", "return",
"import", "from", "as", "try", "except", "finally", "raise",
"with", "yield", "lambda", "pass", "break", "continue",
"and", "or", "not", "in", "is", "del", "global", "nonlocal",
"assert", "async", "await", nullptr
};
for (const char** k = keywords; *k; ++k) {
if (text == *k) return true;
}
return false;
}
static bool isPythonBuiltin(const std::string& text) {
static const char* builtins[] = {
"True", "False", "None", "print", "len", "range", "int",
"str", "float", "list", "dict", "set", "tuple", "type",
"isinstance", "super", "self", nullptr
};
for (const char** b = builtins; *b; ++b) {
if (text == *b) return true;
}
return false;
}
static void walkPython(TSNode node, const std::string& source,
std::vector<HighlightSpan>& spans) {
if (ts_node_is_null(node)) return;
std::string type = nodeType(node);
std::string text = nodeText(node, source);
bool descend = true;
// Leaf / terminal categorization
if (type == "comment") {
addSpan(spans, node, TokenCategory::Comment);
descend = false;
} else if (type == "string" || type == "concatenated_string") {
addSpan(spans, node, TokenCategory::String);
descend = false;
} else if (type == "integer" || type == "float") {
addSpan(spans, node, TokenCategory::Number);
descend = false;
} else if (type == "identifier") {
// Context-dependent: function name, parameter, builtin, or plain
TSNode parent = ts_node_parent(node);
std::string parentType = ts_node_is_null(parent) ? "" : nodeType(parent);
if (parentType == "function_definition") {
TSNode nameField = ts_node_child_by_field_name(parent, "name", 4);
if (!ts_node_is_null(nameField) &&
ts_node_start_byte(nameField) == ts_node_start_byte(node)) {
addSpan(spans, node, TokenCategory::Function);
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
} else if (parentType == "parameters" || parentType == "default_parameter") {
addSpan(spans, node, TokenCategory::Parameter);
} else if (parentType == "call") {
TSNode funcField = ts_node_child_by_field_name(parent, "function", 8);
if (!ts_node_is_null(funcField) &&
ts_node_start_byte(funcField) == ts_node_start_byte(node)) {
addSpan(spans, node, TokenCategory::Function);
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
} else if (parentType == "type") {
addSpan(spans, node, TokenCategory::Type);
} else if (isPythonBuiltin(text)) {
addSpan(spans, node, TokenCategory::Builtin);
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
descend = false;
} else if (type == "type") {
addSpan(spans, node, TokenCategory::Type);
descend = false;
} else if (!ts_node_is_named(node)) {
// Anonymous/unnamed nodes are operators, keywords, punctuation
if (isPythonKeyword(text)) {
addSpan(spans, node, TokenCategory::Keyword);
} else if (text == "(" || text == ")" || text == "[" || text == "]" ||
text == "{" || text == "}" || text == ":" || text == "," ||
text == "." || text == ";") {
addSpan(spans, node, TokenCategory::Punctuation);
} else if (text == "+" || text == "-" || text == "*" || text == "/" ||
text == "%" || text == "=" || text == "==" || text == "!=" ||
text == "<" || text == ">" || text == "<=" || text == ">=" ||
text == "+=" || text == "-=" || text == "*=" || text == "/=" ||
text == "**" || text == "//" || text == "->" || text == "@") {
addSpan(spans, node, TokenCategory::Operator);
}
descend = false;
}
if (descend) {
uint32_t count = ts_node_child_count(node);
for (uint32_t i = 0; i < count; ++i) {
walkPython(ts_node_child(node, i), source, spans);
}
}
}
// --- C++ -----------------------------------------------------------
static bool isCppKeyword(const std::string& text) {
static const char* keywords[] = {
"auto", "break", "case", "catch", "class", "const", "constexpr",
"continue", "default", "delete", "do", "else", "enum", "explicit",
"extern", "for", "friend", "goto", "if", "inline", "mutable",
"namespace", "new", "noexcept", "operator", "private", "protected",
"public", "register", "return", "sizeof", "static", "static_assert",
"static_cast", "struct", "switch", "template", "this", "throw",
"try", "typedef", "typeid", "typename", "union", "using",
"virtual", "volatile", "while", "override", "final",
"co_await", "co_return", "co_yield", "concept", "requires",
"consteval", "constinit", nullptr
};
for (const char** k = keywords; *k; ++k) {
if (text == *k) return true;
}
return false;
}
static bool isCppType(const std::string& text) {
static const char* types[] = {
"void", "bool", "char", "int", "float", "double", "long",
"short", "unsigned", "signed", "size_t", "nullptr_t",
"int8_t", "int16_t", "int32_t", "int64_t",
"uint8_t", "uint16_t", "uint32_t", "uint64_t",
"string", "vector", "map", "set", "array", "tuple",
"unique_ptr", "shared_ptr", "weak_ptr", "optional",
nullptr
};
for (const char** t = types; *t; ++t) {
if (text == *t) return true;
}
return false;
}
static void walkCpp(TSNode node, const std::string& source,
std::vector<HighlightSpan>& spans) {
if (ts_node_is_null(node)) return;
std::string type = nodeType(node);
std::string text = nodeText(node, source);
bool descend = true;
if (type == "comment") {
addSpan(spans, node, TokenCategory::Comment);
descend = false;
} else if (type == "string_literal" || type == "raw_string_literal" ||
type == "char_literal" || type == "string_content") {
addSpan(spans, node, TokenCategory::String);
descend = false;
} else if (type == "number_literal") {
addSpan(spans, node, TokenCategory::Number);
descend = false;
} else if (type == "primitive_type" || type == "sized_type_specifier" ||
type == "type_identifier") {
addSpan(spans, node, TokenCategory::Type);
descend = false;
} else if (type == "identifier") {
TSNode parent = ts_node_parent(node);
std::string parentType = ts_node_is_null(parent) ? "" : nodeType(parent);
if (parentType == "function_declarator") {
TSNode declField = ts_node_child_by_field_name(parent, "declarator", 10);
if (!ts_node_is_null(declField) &&
ts_node_start_byte(declField) == ts_node_start_byte(node)) {
addSpan(spans, node, TokenCategory::Function);
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
} else if (parentType == "call_expression") {
TSNode funcField = ts_node_child_by_field_name(parent, "function", 8);
if (!ts_node_is_null(funcField) &&
ts_node_start_byte(funcField) == ts_node_start_byte(node)) {
addSpan(spans, node, TokenCategory::Function);
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
} else if (parentType == "parameter_declaration") {
TSNode declField = ts_node_child_by_field_name(parent, "declarator", 10);
if (!ts_node_is_null(declField) &&
ts_node_start_byte(declField) == ts_node_start_byte(node)) {
addSpan(spans, node, TokenCategory::Parameter);
} else if (isCppType(text)) {
addSpan(spans, node, TokenCategory::Type);
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
} else if (isCppType(text)) {
addSpan(spans, node, TokenCategory::Type);
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
descend = false;
} else if (type == "true" || type == "false" || type == "nullptr") {
addSpan(spans, node, TokenCategory::Builtin);
descend = false;
} else if (type == "preproc_include" || type == "preproc_def" ||
type == "preproc_ifdef" || type == "preproc_else" ||
type == "preproc_endif" || type == "preproc_call") {
addSpan(spans, node, TokenCategory::Keyword);
descend = false;
} else if (type == "system_lib_string") {
addSpan(spans, node, TokenCategory::String);
descend = false;
} else if (!ts_node_is_named(node)) {
if (isCppKeyword(text)) {
addSpan(spans, node, TokenCategory::Keyword);
} else if (text == "(" || text == ")" || text == "[" || text == "]" ||
text == "{" || text == "}" || text == ";" || text == "," ||
text == "." || text == "::" || text == ":" || text == "->") {
addSpan(spans, node, TokenCategory::Punctuation);
} else if (text == "+" || text == "-" || text == "*" || text == "/" ||
text == "%" || text == "=" || text == "==" || text == "!=" ||
text == "<" || text == ">" || text == "<=" || text == ">=" ||
text == "+=" || text == "-=" || text == "*=" || text == "/=" ||
text == "&&" || text == "||" || text == "!" || text == "&" ||
text == "|" || text == "^" || text == "~" || text == "<<" ||
text == ">>" || text == "++" || text == "--") {
addSpan(spans, node, TokenCategory::Operator);
} else if (isCppType(text)) {
addSpan(spans, node, TokenCategory::Type);
}
descend = false;
}
if (descend) {
uint32_t count = ts_node_child_count(node);
for (uint32_t i = 0; i < count; ++i) {
walkCpp(ts_node_child(node, i), source, spans);
}
}
}
// --- Elisp ---------------------------------------------------------
static bool isElispKeyword(const std::string& text) {
static const char* keywords[] = {
"defun", "defvar", "defconst", "defmacro", "defcustom",
"let", "let*", "if", "when", "unless", "cond", "while",
"dolist", "dotimes", "progn", "prog1", "prog2",
"lambda", "setq", "setf", "require", "provide",
"interactive", "save-excursion", "save-restriction",
"condition-case", "unwind-protect", "catch", "throw",
nullptr
};
for (const char** k = keywords; *k; ++k) {
if (text == *k) return true;
}
return false;
}
static void walkElisp(TSNode node, const std::string& source,
std::vector<HighlightSpan>& spans) {
if (ts_node_is_null(node)) return;
std::string type = nodeType(node);
std::string text = nodeText(node, source);
bool descend = true;
if (type == "comment") {
addSpan(spans, node, TokenCategory::Comment);
descend = false;
} else if (type == "string") {
addSpan(spans, node, TokenCategory::String);
descend = false;
} else if (type == "integer" || type == "float") {
addSpan(spans, node, TokenCategory::Number);
descend = false;
} else if (type == "symbol") {
if (isElispKeyword(text)) {
addSpan(spans, node, TokenCategory::Keyword);
} else if (text.size() > 0 && text[0] == ':') {
// keyword symbol like :test
addSpan(spans, node, TokenCategory::Builtin);
} else if (text == "t" || text == "nil") {
addSpan(spans, node, TokenCategory::Builtin);
} else {
TSNode parent = ts_node_parent(node);
std::string parentType = ts_node_is_null(parent) ? "" : nodeType(parent);
if (parentType == "function_definition" || parentType == "special_form") {
// Check if this is the name position (second element)
TSNode firstSibling = ts_node_named_child(parent, 0);
TSNode secondSibling = ts_node_named_child(parent, 1);
if (!ts_node_is_null(firstSibling) &&
isElispKeyword(nodeText(firstSibling, source)) &&
!ts_node_is_null(secondSibling) &&
ts_node_start_byte(secondSibling) == ts_node_start_byte(node)) {
addSpan(spans, node, TokenCategory::Function);
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
} else {
addSpan(spans, node, TokenCategory::Identifier);
}
}
descend = false;
} else if (!ts_node_is_named(node)) {
if (text == "(" || text == ")" || text == "[" || text == "]") {
addSpan(spans, node, TokenCategory::Punctuation);
} else if (text == "'" || text == "`" || text == ",") {
addSpan(spans, node, TokenCategory::Operator);
}
descend = false;
}
if (descend) {
uint32_t count = ts_node_child_count(node);
for (uint32_t i = 0; i < count; ++i) {
walkElisp(ts_node_child(node, i), source, spans);
}
}
}
};

View File

@@ -0,0 +1,161 @@
// Step 52 TDD Test: Syntax highlighting via tree-sitter
//
// Verifies that SyntaxHighlighter produces correct token spans:
// 1. Python: keywords, strings, comments, numbers, functions, parameters
// 2. C++: keywords, types, strings, comments, functions
// 3. Elisp: keywords, strings, comments, function definitions
// 4. Span coverage: all source bytes covered or categorized
// 5. Unknown language returns empty spans
#include <iostream>
#include <string>
#include <cassert>
#include <vector>
#include <algorithm>
#include "SyntaxHighlighter.h"
static bool hasCategory(const std::vector<HighlightSpan>& spans,
TokenCategory cat) {
return std::any_of(spans.begin(), spans.end(),
[cat](const HighlightSpan& s) { return s.category == cat; });
}
static bool hasCategoryAt(const std::vector<HighlightSpan>& spans,
TokenCategory cat, uint32_t pos) {
return std::any_of(spans.begin(), spans.end(),
[cat, pos](const HighlightSpan& s) {
return s.category == cat && s.start <= pos && pos < s.end;
});
}
static int countCategory(const std::vector<HighlightSpan>& spans,
TokenCategory cat) {
return (int)std::count_if(spans.begin(), spans.end(),
[cat](const HighlightSpan& s) { return s.category == cat; });
}
int main() {
int passed = 0;
int failed = 0;
// --- Test 1: Python keywords and structure ---
{
std::string source = "def greet(name):\n return name\n";
auto spans = SyntaxHighlighter::highlight(source, "python");
assert(!spans.empty() && "Should produce spans for Python");
assert(hasCategory(spans, TokenCategory::Keyword) && "Should have keywords (def, return)");
assert(hasCategory(spans, TokenCategory::Function) && "Should have function name");
assert(hasCategory(spans, TokenCategory::Parameter) && "Should have parameter");
// "def" should be a keyword at position 0
assert(hasCategoryAt(spans, TokenCategory::Keyword, 0) && "'def' at pos 0 should be keyword");
// "greet" should be a function name at position 4
assert(hasCategoryAt(spans, TokenCategory::Function, 4) && "'greet' at pos 4 should be function");
// "name" in parameters should be parameter (position 10)
assert(hasCategoryAt(spans, TokenCategory::Parameter, 10) && "'name' param should be parameter");
std::cout << "Test 1 PASS: Python keywords, functions, parameters" << std::endl;
++passed;
}
// --- Test 2: Python strings and comments ---
{
std::string source = "# a comment\nx = \"hello\"\ny = 42\n";
auto spans = SyntaxHighlighter::highlight(source, "python");
assert(hasCategory(spans, TokenCategory::Comment) && "Should have comment");
assert(hasCategory(spans, TokenCategory::String) && "Should have string");
assert(hasCategory(spans, TokenCategory::Number) && "Should have number");
// Comment starts at position 0
assert(hasCategoryAt(spans, TokenCategory::Comment, 0) && "Comment at pos 0");
std::cout << "Test 2 PASS: Python strings, comments, numbers" << std::endl;
++passed;
}
// --- Test 3: C++ keywords and types ---
{
std::string source = "int add(int a, int b) { return a + b; }";
auto spans = SyntaxHighlighter::highlight(source, "cpp");
assert(!spans.empty() && "Should produce spans for C++");
assert(hasCategory(spans, TokenCategory::Type) && "Should have type (int)");
assert(hasCategory(spans, TokenCategory::Keyword) && "Should have keyword (return)");
assert(hasCategory(spans, TokenCategory::Operator) && "Should have operator (+)");
std::cout << "Test 3 PASS: C++ keywords, types, operators" << std::endl;
++passed;
}
// --- Test 4: C++ strings and comments ---
{
std::string source = "// line comment\nstd::string s = \"hello\";\n";
auto spans = SyntaxHighlighter::highlight(source, "cpp");
assert(hasCategory(spans, TokenCategory::Comment) && "Should have comment");
assert(hasCategory(spans, TokenCategory::String) && "Should have string");
assert(hasCategoryAt(spans, TokenCategory::Comment, 0) && "Comment at pos 0");
std::cout << "Test 4 PASS: C++ strings and comments" << std::endl;
++passed;
}
// --- Test 5: Elisp keywords and structure ---
{
std::string source = "(defun my-func (x)\n (+ x 1))\n";
auto spans = SyntaxHighlighter::highlight(source, "elisp");
assert(!spans.empty() && "Should produce spans for Elisp");
assert(hasCategory(spans, TokenCategory::Keyword) && "Should have keyword (defun)");
assert(hasCategory(spans, TokenCategory::Number) && "Should have number (1)");
assert(hasCategory(spans, TokenCategory::Punctuation) && "Should have punctuation (parens)");
std::cout << "Test 5 PASS: Elisp keywords, numbers, punctuation" << std::endl;
++passed;
}
// --- Test 6: Empty and unknown language ---
{
auto empty = SyntaxHighlighter::highlight("", "python");
assert(empty.empty() && "Empty source should produce no spans");
auto unknown = SyntaxHighlighter::highlight("hello world", "brainfuck");
assert(unknown.empty() && "Unknown language should produce no spans");
std::cout << "Test 6 PASS: Empty and unknown language handling" << std::endl;
++passed;
}
// --- Test 7: Spans are sorted by position ---
{
std::string source = "def f(x):\n return x + 1\n";
auto spans = SyntaxHighlighter::highlight(source, "python");
for (size_t i = 1; i < spans.size(); ++i) {
assert(spans[i].start >= spans[i-1].start &&
"Spans should be sorted by start position");
}
std::cout << "Test 7 PASS: Spans are sorted" << std::endl;
++passed;
}
// --- Test 8: categoryName returns valid strings ---
{
assert(std::string(SyntaxHighlighter::categoryName(TokenCategory::Keyword)) == "keyword");
assert(std::string(SyntaxHighlighter::categoryName(TokenCategory::String)) == "string");
assert(std::string(SyntaxHighlighter::categoryName(TokenCategory::Comment)) == "comment");
assert(std::string(SyntaxHighlighter::categoryName(TokenCategory::Function)) == "function");
std::cout << "Test 8 PASS: categoryName returns correct names" << std::endl;
++passed;
}
// --- Summary ---
std::cout << "\n=== Step 52 Results: " << passed << " passed, " << failed << " failed ===" << std::endl;
return failed > 0 ? 1 : 0;
}

View File

@@ -0,0 +1,182 @@
// Step 54 TDD Test: Configurable keybinding profiles
//
// Verifies that KeybindingManager:
// 1. Defaults to VSCode profile
// 2. Can switch between profiles (VSCode, JetBrains, Emacs)
// 3. Different profiles have different bindings for the same actions
// 4. Reverse lookup (key combo -> action) works
// 5. Custom binding overrides work
// 6. All profiles define core actions
// 7. Profile names round-trip correctly
#include <iostream>
#include <string>
#include <cassert>
#include <algorithm>
#include "KeybindingManager.h"
int main() {
int passed = 0;
int failed = 0;
// --- Test 1: Default profile is VSCode ---
{
KeybindingManager mgr;
assert(mgr.getProfile() == KeybindingProfile::VSCode &&
"Default profile should be VSCode");
std::cout << "Test 1 PASS: Default profile is VSCode" << std::endl;
++passed;
}
// --- Test 2: VSCode bindings are correct ---
{
KeybindingManager mgr;
KeyCombo save = mgr.getBinding("file.save");
assert(save.key == 'S' && save.modifiers == MOD_CTRL &&
"VSCode: file.save should be Ctrl+S");
KeyCombo undo = mgr.getBinding("edit.undo");
assert(undo.key == 'Z' && undo.modifiers == MOD_CTRL &&
"VSCode: edit.undo should be Ctrl+Z");
KeyCombo redo = mgr.getBinding("edit.redo");
assert(redo.key == 'Y' && redo.modifiers == MOD_CTRL &&
"VSCode: edit.redo should be Ctrl+Y");
std::cout << "Test 2 PASS: VSCode bindings correct" << std::endl;
++passed;
}
// --- Test 3: JetBrains has different redo ---
{
KeybindingManager mgr;
mgr.setProfile(KeybindingProfile::JetBrains);
assert(mgr.getProfile() == KeybindingProfile::JetBrains);
KeyCombo redo = mgr.getBinding("edit.redo");
assert(redo.key == 'Z' && redo.modifiers == (MOD_CTRL | MOD_SHIFT) &&
"JetBrains: edit.redo should be Ctrl+Shift+Z");
// But save is still Ctrl+S
KeyCombo save = mgr.getBinding("file.save");
assert(save.key == 'S' && save.modifiers == MOD_CTRL &&
"JetBrains: file.save should still be Ctrl+S");
std::cout << "Test 3 PASS: JetBrains profile has different redo" << std::endl;
++passed;
}
// --- Test 4: Emacs profile loads ---
{
KeybindingManager mgr;
mgr.setProfile(KeybindingProfile::Emacs);
assert(mgr.getProfile() == KeybindingProfile::Emacs);
// Emacs uses Ctrl+Y for paste
KeyCombo paste = mgr.getBinding("edit.paste");
assert(paste.key == 'Y' && paste.modifiers == MOD_CTRL &&
"Emacs: edit.paste should be Ctrl+Y");
std::cout << "Test 4 PASS: Emacs profile loads with correct bindings" << std::endl;
++passed;
}
// --- Test 5: Reverse lookup (combo -> action) ---
{
KeybindingManager mgr; // VSCode
KeyCombo ctrlS = {'S', MOD_CTRL};
std::string action = mgr.getAction(ctrlS);
assert(action == "file.save" && "Ctrl+S should map to file.save");
KeyCombo unknown = {'Q', MOD_CTRL | MOD_ALT | MOD_SHIFT};
std::string none = mgr.getAction(unknown);
assert(none.empty() && "Unknown combo should return empty string");
std::cout << "Test 5 PASS: Reverse lookup works" << std::endl;
++passed;
}
// --- Test 6: Custom binding override ---
{
KeybindingManager mgr;
mgr.setBinding("file.save", {'S', MOD_CTRL | MOD_ALT});
KeyCombo save = mgr.getBinding("file.save");
assert(save.key == 'S' && save.modifiers == (MOD_CTRL | MOD_ALT) &&
"Custom override should change binding");
std::cout << "Test 6 PASS: Custom binding override" << std::endl;
++passed;
}
// --- Test 7: All profiles define core actions ---
{
std::vector<std::string> coreActions = {
"file.save", "edit.undo", "edit.redo", "edit.copy", "edit.paste",
"search.find", "code.comment"
};
for (auto profile : KeybindingManager::availableProfiles()) {
KeybindingManager mgr;
mgr.setProfile(profile);
for (const auto& action : coreActions) {
KeyCombo combo = mgr.getBinding(action);
assert(combo.key != 0 &&
(std::string("Profile ") +
KeybindingManager::profileName(profile) +
" missing action: " + action).c_str());
}
}
std::cout << "Test 7 PASS: All profiles define core actions" << std::endl;
++passed;
}
// --- Test 8: Profile name round-trip ---
{
for (auto profile : KeybindingManager::availableProfiles()) {
const char* name = KeybindingManager::profileName(profile);
KeybindingProfile roundTripped = KeybindingManager::profileFromName(name);
assert(roundTripped == profile && "Profile name should round-trip");
}
// Unknown name defaults to VSCode
KeybindingProfile def = KeybindingManager::profileFromName("unknown");
assert(def == KeybindingProfile::VSCode && "Unknown profile should default to VSCode");
std::cout << "Test 8 PASS: Profile name round-trip" << std::endl;
++passed;
}
// --- Test 9: KeyCombo toString ---
{
KeyCombo ctrlS = {'S', MOD_CTRL};
assert(ctrlS.toString() == "Ctrl+S" && "Ctrl+S toString");
KeyCombo ctrlShiftZ = {'Z', MOD_CTRL | MOD_SHIFT};
assert(ctrlShiftZ.toString() == "Ctrl+Shift+Z" && "Ctrl+Shift+Z toString");
std::cout << "Test 9 PASS: KeyCombo toString" << std::endl;
++passed;
}
// --- Test 10: listActions returns all bound actions ---
{
KeybindingManager mgr;
auto actions = mgr.listActions();
assert(actions.size() > 20 && "VSCode profile should have 20+ actions");
// Check a few exist
assert(std::find(actions.begin(), actions.end(), "file.save") != actions.end());
assert(std::find(actions.begin(), actions.end(), "edit.undo") != actions.end());
assert(std::find(actions.begin(), actions.end(), "search.find") != actions.end());
std::cout << "Test 10 PASS: listActions returns all bound actions" << std::endl;
++passed;
}
// --- Summary ---
std::cout << "\n=== Step 54 Results: " << passed << " passed, " << failed << " failed ===" << std::endl;
return failed > 0 ? 1 : 0;
}