Fix cross-project target grounding and add handoff note

This commit is contained in:
Bill
2026-03-08 10:53:30 -06:00
parent 9f9d9590c3
commit a09fc55325
6 changed files with 670 additions and 22 deletions

View File

@@ -0,0 +1,81 @@
# Agent Handoff - 2026-03-08 - Cross-Project Target Grounding (GR-027)
## Scope
This handoff captures work completed on the open gap `GR-027` (cross-project task targeting mismatch) in Whetstone taskitem generation/validation.
## Problem Addressed
Cross-repo orchestration specs were producing Whetstone-internal `targetFiles` (`editor/src/...`, `tools/mcp/...`) even when the workspace was set to `/home/bill/Documents` and the spec explicitly targeted external repos.
## What Was Implemented
### 1) Generator/Queue cross-project mismatch detection
File: `editor/src/mcp/RegisterArchitectIntakeTools.h`
- Added cross-project repo signal detection from normalized requirements.
- Added cross-project fallback target grounding when strict execution contract is enabled and `targetFiles` are otherwise missing.
- Added queue blocker `cross_project_target_mismatch` when cross-project intent is detected but generated targets are internal-only.
- Added blocker metadata mapping for `cross_project_target_mismatch` to gap class `cross_project_targeting_gap` and remediation hints.
### 2) Validator mismatch classification
File: `editor/src/mcp/RegisterValidationTools.h`
- Extended `whetstone_validate_taskitem` schema with optional `expected_project_roots`.
- Added cross-project mismatch detection in validation results.
- Added gap class mapping `cross_project_targeting_gap` and recommended tool routing for that class.
### 3) Semantic artifact-file injection into pipeline
File: `tools/mcp/run_sprint_taskitem_pipeline.sh`
- Extended semantic requirement injection to derive `requirement` entries from contract `required_artifacts` backtick paths.
- Injected normalized requirements like `explicit target file: cad_orchestrator/models.py`.
### 4) Path extraction/normalization hardening
File: `editor/src/mcp/RegisterArchitectIntakeTools.h`
- Added normalization helpers for path-like tokens (including absolute `/home/bill/Documents/...` inputs).
- Added extraction of inline path tokens beyond code-fenced/backticked fragments.
### 5) Per-task target scoping for requirement-slice tasks
File: `editor/src/mcp/RegisterArchitectIntakeTools.h`
- Added task-level scoping so tasks titled `Requirement Slice semantic-artifact-file-*` receive only their corresponding single target file.
- Non-slice tasks currently keep the broader target set.
### 6) Tests updated
Files:
- `editor/tests/step615_test.cpp`
- `editor/tests/step682_test.cpp`
- Added focused checks for cross-project target grounding and validation mismatch classification.
## Key Validation Runs
### A) Cross-project roadmap run
- Run: `logs/taskitem_runs/constcad_ecosystem_roadmap_20260308_093527`
- Result: no internal default fallback; `targetFiles` grounded to cross-project targets.
### B) SPRINT-005 run (pre file-level injection)
- Run: `logs/taskitem_runs/sprint_005_execution_spec_whetstone_20260308_093527`
- Result: repo-level targets (README-level grounding).
### C) SPRINT-005 run (after file-level semantic injection)
- Run: `logs/taskitem_runs/sprint_005_execution_spec_whetstone_20260308_101705`
- Result: concrete cross-project files propagated into `targetFiles`.
### D) SPRINT-005 run (after per-task scoping)
- Run: `logs/taskitem_runs/sprint_005_execution_spec_whetstone_20260308_103102`
- Result:
- `Requirement Slice semantic-artifact-file-*` tasks have single-file `targetFiles`.
- Composite/review tasks still carry broader file lists.
## Current Status
- `GR-027` core mismatch is addressed in generation, queue gating, and validation classification.
- Cross-project explicit file grounding is now present for hard specs that include artifact paths.
- Requirement-slice tasks are now narrowly scoped.
## Suggested Next Steps
1. Scope non-slice tasks (`Intake Foundation`, `Execution Readiness`, `Architect Review`) into smaller deterministic file groups.
2. Add/refresh dedicated regression tests for:
- queue blocker `cross_project_target_mismatch`
- per-task single-file scoping behavior for semantic artifact slices.
3. Update `docs/generator_readiness_gap_registry_2026-02-26.md` to set `GR-027` from `open` to `partial` or `done` after final acceptance criteria decision.

View File

@@ -282,11 +282,81 @@
}; };
} }
static std::string trimPathToken(const std::string& in) {
if (in.empty()) return "";
std::size_t start = 0;
std::size_t end = in.size();
auto isTrimChar = [](char c) {
return c == '"' || c == '\'' || c == '`' || c == ',' || c == ';' || c == ':' ||
c == '(' || c == ')' || c == '[' || c == ']' || c == '{' || c == '}';
};
while (start < end && (in[start] == ' ' || in[start] == '\t' || isTrimChar(in[start]))) ++start;
while (end > start && (in[end - 1] == ' ' || in[end - 1] == '\t' || isTrimChar(in[end - 1]))) --end;
return in.substr(start, end - start);
}
static bool looksLikePathToken(const std::string& token) {
if (token.empty()) return false;
if (token.find(' ') != std::string::npos) return false;
const bool hasSlash = token.find('/') != std::string::npos;
const bool hasDot = token.find('.') != std::string::npos;
if (!hasSlash || !hasDot) return false;
if (token.rfind("http://", 0) == 0 || token.rfind("https://", 0) == 0) return false;
if (token.rfind("wstone_", 0) == 0 || token.rfind("cd ", 0) == 0) return false;
if (token.find('<') != std::string::npos || token.find('>') != std::string::npos) return false;
return true;
}
static std::string normalizeTargetFileCandidate(const std::string& raw) {
std::string value = trimPathToken(raw);
if (value.empty()) return "";
if (value.rfind("./", 0) == 0) value = value.substr(2);
const std::string homeDocsPrefix = "/home/bill/Documents/";
if (value.rfind(homeDocsPrefix, 0) == 0) {
value = value.substr(homeDocsPrefix.size());
}
if (value.rfind("CLionProjects/whetstone_DSL/", 0) == 0) {
value = value.substr(std::string("CLionProjects/whetstone_DSL/").size());
}
if (!looksLikePathToken(value)) return "";
return value;
}
static std::vector<std::string> extractPathTokensFromText(const std::string& text) {
std::vector<std::string> out;
std::string token;
auto flush = [&out, &token]() {
if (token.empty()) return;
const std::string normalized = normalizeTargetFileCandidate(token);
if (!normalized.empty()) appendUnique(&out, normalized);
token.clear();
};
for (char c : text) {
if (c == ' ' || c == '\n' || c == '\t' || c == ',' || c == ';' || c == '(' || c == ')') {
flush();
} else {
token.push_back(c);
}
}
flush();
return out;
}
static json collectExecutionHints(const std::vector<NormalizedRequirement>& requirements) { static json collectExecutionHints(const std::vector<NormalizedRequirement>& requirements) {
std::vector<std::string> stepIds; std::vector<std::string> stepIds;
std::vector<std::string> files; std::vector<std::string> files;
std::vector<std::string> tools; std::vector<std::string> tools;
std::vector<std::string> commands; std::vector<std::string> commands;
std::vector<std::string> repoSignals;
const std::vector<std::string> knownCrossProjectRepos = {
"cad_orchestrator",
"hivemind",
"gleaner",
"constcad",
"whetstone_planning_tool",
"whetstone_ubuntu_frontend_tool"
};
for (const auto& requirement : requirements) { for (const auto& requirement : requirements) {
const std::string text = requirement.normalizedText; const std::string text = requirement.normalizedText;
@@ -317,12 +387,14 @@
std::size_t end = text.find('`', pos + 1); std::size_t end = text.find('`', pos + 1);
if (end == std::string::npos) break; if (end == std::string::npos) break;
std::string candidate = text.substr(pos + 1, end - pos - 1); std::string candidate = text.substr(pos + 1, end - pos - 1);
if (candidate.find('/') != std::string::npos || candidate.find('.') != std::string::npos) { std::string normalizedPath = normalizeTargetFileCandidate(candidate);
appendUnique(&files, candidate); if (!normalizedPath.empty()) appendUnique(&files, normalizedPath);
}
pos = end + 1; pos = end + 1;
} }
const auto inlinePathTokens = extractPathTokensFromText(text);
for (const auto& p : inlinePathTokens) appendUnique(&files, p);
pos = 0; pos = 0;
while ((pos = lower.find("step", pos)) != std::string::npos) { while ((pos = lower.find("step", pos)) != std::string::npos) {
std::size_t end = pos + 4; std::size_t end = pos + 4;
@@ -339,16 +411,123 @@
if (lower.find("go test") != std::string::npos) appendUnique(&commands, "go test ./..."); if (lower.find("go test") != std::string::npos) appendUnique(&commands, "go test ./...");
if (lower.find("npm test") != std::string::npos) appendUnique(&commands, "npm test"); if (lower.find("npm test") != std::string::npos) appendUnique(&commands, "npm test");
if (lower.find("cmake --build") != std::string::npos) appendUnique(&commands, "cmake --build ."); if (lower.find("cmake --build") != std::string::npos) appendUnique(&commands, "cmake --build .");
for (const auto& repo : knownCrossProjectRepos) {
if (lower.find(repo) != std::string::npos) appendUnique(&repoSignals, repo);
}
} }
return { return {
{"stepIds", sortedUnique(stepIds)}, {"stepIds", sortedUnique(stepIds)},
{"targetFiles", sortedUnique(files)}, {"targetFiles", sortedUnique(files)},
{"requiredTools", sortedUnique(tools)}, {"requiredTools", sortedUnique(tools)},
{"acceptanceCommands", sortedUnique(commands)} {"acceptanceCommands", sortedUnique(commands)},
{"crossProjectRepos", sortedUnique(repoSignals)}
}; };
} }
static bool isWhetstoneInternalPath(const std::string& path) {
const std::string lower = intakeToLowerCopy(path);
return lower.rfind("editor/", 0) == 0 ||
lower.rfind("tools/", 0) == 0 ||
lower.rfind("logs/", 0) == 0;
}
static bool isKnownCrossProjectRepo(const std::string& value) {
const std::string lower = intakeToLowerCopy(value);
return lower == "cad_orchestrator" ||
lower == "hivemind" ||
lower == "gleaner" ||
lower == "constcad" ||
lower == "whetstone_planning_tool" ||
lower == "whetstone_ubuntu_frontend_tool";
}
static json inferCrossProjectTargetsFromHints(const json& hints) {
std::vector<std::string> targets;
const json repos = hints.value("crossProjectRepos", json::array());
if (!repos.is_array()) return json::array();
for (const auto& repoJson : repos) {
if (!repoJson.is_string()) continue;
const std::string repo = intakeToLowerCopy(repoJson.get<std::string>());
if (!isKnownCrossProjectRepo(repo)) continue;
appendUnique(&targets, repo + "/README.md");
}
return sortedUnique(targets);
}
static bool hasCrossProjectIntent(const std::vector<NormalizedRequirement>& requirements) {
int hitCount = 0;
for (const auto& requirement : requirements) {
const std::string lower = intakeToLowerCopy(requirement.normalizedText);
if (lower.find("cross-repo") != std::string::npos ||
lower.find("multi-project") != std::string::npos ||
lower.find("multi-repo") != std::string::npos ||
lower.find("ecosystem") != std::string::npos) {
return true;
}
if (lower.find("cad_orchestrator") != std::string::npos) ++hitCount;
if (lower.find("hivemind") != std::string::npos) ++hitCount;
if (lower.find("gleaner") != std::string::npos) ++hitCount;
if (lower.find("constcad") != std::string::npos) ++hitCount;
if (hitCount >= 2) return true;
}
return false;
}
static bool hasOnlyInternalTargets(const json& tasks) {
if (!tasks.is_array() || tasks.empty()) return true;
bool sawTarget = false;
for (const auto& task : tasks) {
if (!task.is_object()) continue;
const json contract = task.value("executionContract", json::object());
const json files = contract.value("targetFiles", json::array());
if (!files.is_array()) continue;
for (const auto& fileJson : files) {
if (!fileJson.is_string()) continue;
sawTarget = true;
if (!isWhetstoneInternalPath(fileJson.get<std::string>())) return false;
}
}
return sawTarget;
}
static std::string parseExplicitTargetFileFromRequirementText(const std::string& text) {
const std::string prefix = "explicit target file:";
const std::string lower = intakeToLowerCopy(text);
if (lower.rfind(prefix, 0) != 0) return "";
std::string raw = text.substr(prefix.size());
return normalizeTargetFileCandidate(raw);
}
static std::string requirementIdFromSliceTitle(const std::string& title) {
const std::string prefix = "Requirement Slice ";
if (title.rfind(prefix, 0) != 0) return "";
std::string id = title.substr(prefix.size());
return trimPathToken(id);
}
static json selectTaskScopedTargetFiles(const AnnotatedTaskitem& task,
const std::vector<NormalizedRequirement>& requirements,
const json& defaultTargetFiles) {
std::vector<std::string> scoped;
const std::string sliceRequirementId = requirementIdFromSliceTitle(task.base.title);
for (const auto& requirement : requirements) {
const std::string file = parseExplicitTargetFileFromRequirementText(requirement.normalizedText);
if (file.empty()) continue;
const bool exactMatch = !sliceRequirementId.empty() && requirement.requirementId == sliceRequirementId;
const bool containsMatch = !sliceRequirementId.empty() &&
task.base.title.find(requirement.requirementId) != std::string::npos;
if (exactMatch || containsMatch) {
appendUnique(&scoped, file);
}
}
if (scoped.empty()) return defaultTargetFiles;
json out = json::array();
for (const auto& file : sortedUnique(scoped)) out.push_back(file);
return out;
}
static json inferResourceLocks(const AnnotatedTaskitem& task) { static json inferResourceLocks(const AnnotatedTaskitem& task) {
std::vector<std::string> locks; std::vector<std::string> locks;
const std::string title = intakeToLowerCopy(task.base.title); const std::string title = intakeToLowerCopy(task.base.title);
@@ -407,12 +586,17 @@
}); });
} }
if (!targetFiles.is_array() || targetFiles.empty()) { if (!targetFiles.is_array() || targetFiles.empty()) {
json crossProjectTargets = inferCrossProjectTargetsFromHints(hints);
if (crossProjectTargets.is_array() && !crossProjectTargets.empty()) {
targetFiles = crossProjectTargets;
} else {
targetFiles = json::array({ targetFiles = json::array({
"editor/src/mcp/RegisterArchitectIntakeTools.h", "editor/src/mcp/RegisterArchitectIntakeTools.h",
"editor/src/mcp/RegisterValidationTools.h", "editor/src/mcp/RegisterValidationTools.h",
"tools/mcp/run_sprint_taskitem_pipeline.sh" "tools/mcp/run_sprint_taskitem_pipeline.sh"
}); });
} }
}
if (!stepIds.is_array() || stepIds.empty()) { if (!stepIds.is_array() || stepIds.empty()) {
stepIds = json::array({"1939", "1940", "1941"}); stepIds = json::array({"1939", "1940", "1941"});
} }
@@ -578,7 +762,10 @@
{"escalate", task.escalate}, {"escalate", task.escalate},
{"reasons", task.reasons} {"reasons", task.reasons}
}; };
json contract = buildExecutionContract(task, executionHints, strictExecutionContract); json taskExecutionHints = executionHints;
taskExecutionHints["targetFiles"] = selectTaskScopedTargetFiles(task, normalized.requirements,
executionHints.value("targetFiles", json::array()));
json contract = buildExecutionContract(task, taskExecutionHints, strictExecutionContract);
if (libraryDispatchEnabled) { if (libraryDispatchEnabled) {
std::string dispatchText = task.base.title; std::string dispatchText = task.base.title;
if (!libraryDispatchContext.empty()) { if (!libraryDispatchContext.empty()) {
@@ -893,6 +1080,10 @@
if (escalateCount > 0) blockers.push_back("escalations_present"); if (escalateCount > 0) blockers.push_back("escalations_present");
if (strictExecutionContract && executionContractMissingCount > 0) blockers.push_back("missing_execution_contract"); if (strictExecutionContract && executionContractMissingCount > 0) blockers.push_back("missing_execution_contract");
if (strictExecutionContract && executionSpecificityLowCount > 0) blockers.push_back("execution_specificity_low"); if (strictExecutionContract && executionSpecificityLowCount > 0) blockers.push_back("execution_specificity_low");
if (strictExecutionContract && hasCrossProjectIntent(requirements) && hasOnlyInternalTargets(args.value("tasks", json::array()))) {
blockers.push_back("cross_project_target_mismatch");
queueWarnings.push_back("cross_project_target_mismatch");
}
bool capabilityGap = false; bool capabilityGap = false;
if (args.contains("capabilitySignals") && args["capabilitySignals"].is_object()) { if (args.contains("capabilitySignals") && args["capabilitySignals"].is_object()) {
const json& signals = args["capabilitySignals"]; const json& signals = args["capabilitySignals"];
@@ -948,9 +1139,11 @@
{"debug_loop_call_plan_fingerprint", debugLoopCallPlan.value("fingerprint", "")}, {"debug_loop_call_plan_fingerprint", debugLoopCallPlan.value("fingerprint", "")},
{"next_action", dominantGapClass == "under_constrained_taskitems" {"next_action", dominantGapClass == "under_constrained_taskitems"
? "tighten_taskitem_generation_contract" ? "tighten_taskitem_generation_contract"
: (dominantGapClass == "cross_project_targeting_gap"
? "regenerate_with_explicit_cross_project_targets"
: (dominantGapClass == "generator_capability_gap" : (dominantGapClass == "generator_capability_gap"
? "run_debug_or_codegen_capability_loop" ? "run_debug_or_codegen_capability_loop"
: "continue_with_validation_and_queue_checks")} : "continue_with_validation_and_queue_checks"))}
}; };
return { return {
@@ -1159,6 +1352,18 @@
})} })}
}; };
} }
if (blocker == "cross_project_target_mismatch") {
return {
{"blocker", blocker},
{"gap_class", "cross_project_targeting_gap"},
{"likely_root_cause", "cross-project intent detected but generated target files resolve only to whetstone-internal paths"},
{"recommended_tools", json::array({
"whetstone_architect_intake",
"whetstone_generate_taskitems",
"whetstone_queue_ready"
})}
};
}
return { return {
{"blocker", blocker}, {"blocker", blocker},
{"gap_class", "unspecified"}, {"gap_class", "unspecified"},

View File

@@ -7,7 +7,10 @@
#include "../TaskitemQualityAuditor.h" #include "../TaskitemQualityAuditor.h"
#include <algorithm>
#include <filesystem> #include <filesystem>
#include <map>
#include <cctype>
void registerValidationTools() { void registerValidationTools() {
tools_.push_back({"whetstone_validate_taskitem", tools_.push_back({"whetstone_validate_taskitem",
@@ -22,8 +25,18 @@
{"prerequisite_ops", {{"type", "array"}, {"items", {{"type", "string"}}}}}, {"prerequisite_ops", {{"type", "array"}, {"items", {{"type", "string"}}}}},
{"reasons", {{"type", "array"}, {"items", {{"type", "string"}}}}}, {"reasons", {{"type", "array"}, {"items", {{"type", "string"}}}}},
{"confidence", {{"type", "integer"}}}, {"confidence", {{"type", "integer"}}},
{"dependency_task_ids", {{"type", "array"}, {"items", {{"type", "string"}}}}} {"dependency_task_ids", {{"type", "array"}, {"items", {{"type", "string"}}}}},
{"execution_contract", {{"type", "object"},
{"description", "Optional execution contract metadata from task generation."}}},
{"resource_locks", {{"type", "array"}, {"items", {{"type", "string"}}},
{"description", "Optional resource lock names for mutual exclusion."}}},
{"capability_signals", {{"type", "object"},
{"description", "Optional capability failure signals (compile/test/toolchain)."}}}
}}, {"required", json::array({"task_id"})}}}}}, }}, {"required", json::array({"task_id"})}}}}},
{"expected_project_roots", {{"type", "array"}, {"items", {{"type", "string"}}},
{"description", "Optional expected external project roots for cross-project target validation."}}},
{"strict_execution_contract", {{"type", "boolean"},
{"description", "Apply strict execution-contract scoring."}}},
{"workspace", {{"type", "string"}, {"workspace", {{"type", "string"},
{"description", "Workspace root path for file resolution."}}} {"description", "Workspace root path for file resolution."}}}
}}, {"required", json::array({"taskitems"})}} }}, {"required", json::array({"taskitems"})}}
@@ -34,12 +47,172 @@
}; };
} }
static std::string classifyValidationIssueGapClass(const std::string& issue) {
if (issue.find("executionSpecificityScore") != std::string::npos ||
issue.find("execution contract") != std::string::npos ||
issue.find("reasons has fewer than 3") != std::string::npos) {
return "under_constrained_taskitem";
}
if (issue.find("cross_project_target_mismatch") != std::string::npos) {
return "cross_project_targeting_gap";
}
if (issue.find("capability_signals indicate") != std::string::npos) {
return "generator_capability_gap";
}
if (issue.find("unresolved file paths") != std::string::npos ||
issue.find("prerequisiteOps missing") != std::string::npos) {
return "environment_or_dependency_gap";
}
if (issue.find("dependencyTaskIds not empty") != std::string::npos) {
return "task_dependency_or_planning_gap";
}
return "generator_capability_gap";
}
static json recommendedToolsForValidationGap(const std::string& gapClass) {
if (gapClass == "under_constrained_taskitem") {
return json::array({"whetstone_generate_taskitems", "whetstone_queue_ready", "whetstone_validate_taskitem"});
}
if (gapClass == "environment_or_dependency_gap") {
return json::array({"whetstone_probe_toolchain_provider", "whetstone_normalize_diagnostics", "whetstone_validate_taskitem"});
}
if (gapClass == "task_dependency_or_planning_gap") {
return json::array({"whetstone_generate_taskitems", "whetstone_queue_ready"});
}
if (gapClass == "resource_lock_semantics_gap") {
return json::array({"whetstone_generate_taskitems", "whetstone_queue_ready", "whetstone_validate_taskitem"});
}
if (gapClass == "cross_project_targeting_gap") {
return json::array({"whetstone_architect_intake", "whetstone_generate_taskitems", "whetstone_queue_ready"});
}
return json::array({
"whetstone_probe_toolchain_provider",
"whetstone_capture_failure_packet",
"whetstone_debug_until_green"
});
}
static std::string lowerCopyValidation(const std::string& in) {
std::string out = in;
std::transform(out.begin(), out.end(), out.begin(), [](unsigned char c) {
return static_cast<char>(std::tolower(c));
});
return out;
}
static bool isInternalTargetPath(const std::string& path) {
const std::string lower = lowerCopyValidation(path);
return lower.rfind("editor/", 0) == 0 ||
lower.rfind("tools/", 0) == 0 ||
lower.rfind("logs/", 0) == 0;
}
static bool hasCrossProjectTargetMismatch(const json& targetFiles,
const std::vector<std::string>& expectedProjectRoots) {
if (expectedProjectRoots.empty() || !targetFiles.is_array() || targetFiles.empty()) return false;
bool sawTarget = false;
for (const auto& target : targetFiles) {
if (!target.is_string()) continue;
sawTarget = true;
const std::string path = lowerCopyValidation(target.get<std::string>());
if (isInternalTargetPath(path)) continue;
for (const auto& root : expectedProjectRoots) {
if (root.empty()) continue;
const std::string prefix = lowerCopyValidation(root) + "/";
if (path.rfind(prefix, 0) == 0) return false;
}
}
return sawTarget;
}
static std::string stableValidationDigest(const std::string& input) {
unsigned long long state = 1469598103934665603ull;
for (unsigned char c : input) {
state ^= static_cast<unsigned long long>(c);
state *= 1099511628211ull;
}
const char* hex = "0123456789abcdef";
std::string out(16, '0');
for (int i = 15; i >= 0; --i) {
out[static_cast<std::size_t>(i)] = hex[state & 0x0full];
state >>= 4;
}
return out;
}
static json buildCapabilityGapCallPlan(bool hasCapabilityGap, bool strictExecutionContract) {
if (!hasCapabilityGap) {
return {
{"enabled", false},
{"profile", "none"},
{"calls", json::array()},
{"fingerprint", stableValidationDigest("none")}
};
}
json calls = json::array({
{
{"tool", "whetstone_probe_toolchain_provider"},
{"args", {
{"verify_provider", true},
{"verify_compiler", true},
{"strict_execution_contract", strictExecutionContract}
}}
},
{
{"tool", "whetstone_capture_failure_packet"},
{"args", {
{"include_compile_logs", true},
{"include_test_logs", true}
}}
},
{
{"tool", "whetstone_debug_until_green"},
{"args", {
{"max_iterations", 3},
{"enforce_determinism", true}
}}
}
});
return {
{"enabled", true},
{"profile", "generator_capability_gap_debug_loop_v1"},
{"calls", calls},
{"fingerprint", stableValidationDigest(calls.dump() + (strictExecutionContract ? "|strict=1" : "|strict=0"))}
};
}
static bool resourceLockLooksValid(const std::string& lockName) {
if (lockName.empty()) return false;
for (char c : lockName) {
const bool ok = (c >= 'a' && c <= 'z') ||
(c >= 'A' && c <= 'Z') ||
(c >= '0' && c <= '9') ||
c == ':' || c == '-' || c == '_' || c == '/' || c == '.';
if (!ok) return false;
}
return true;
}
json runValidateTaskitem(const json& args) { json runValidateTaskitem(const json& args) {
if (!args.contains("taskitems") || !args["taskitems"].is_array()) { if (!args.contains("taskitems") || !args["taskitems"].is_array()) {
return {{"success", false}, {"error", "taskitems_missing_or_invalid"}}; return {{"success", false}, {"error", "taskitems_missing_or_invalid"}};
} }
std::vector<std::string> expectedProjectRoots;
if (args.contains("expected_project_roots")) {
if (!args["expected_project_roots"].is_array()) {
return {{"success", false}, {"error", "expected_project_roots_not_array"}};
}
for (const auto& root : args["expected_project_roots"]) {
if (!root.is_string()) continue;
std::string value = lowerCopyValidation(root.get<std::string>());
if (value.empty()) continue;
expectedProjectRoots.push_back(value);
}
}
std::vector<TaskitemInput> items; std::vector<TaskitemInput> items;
const bool strictExecutionContract = args.value("strict_execution_contract", false);
for (const auto& itemJson : args["taskitems"]) { for (const auto& itemJson : args["taskitems"]) {
if (!itemJson.is_object() || !itemJson.contains("task_id") || !itemJson["task_id"].is_string()) { if (!itemJson.is_object() || !itemJson.contains("task_id") || !itemJson["task_id"].is_string()) {
return {{"success", false}, {"error", "task_id_missing"}}; return {{"success", false}, {"error", "task_id_missing"}};
@@ -65,6 +238,19 @@
if (v.is_string()) item.dependencyTaskIds.push_back(v.get<std::string>()); if (v.is_string()) item.dependencyTaskIds.push_back(v.get<std::string>());
} }
} }
if (itemJson.contains("execution_contract") && itemJson["execution_contract"].is_object()) {
const json& contract = itemJson["execution_contract"];
item.executionSpecificityScore = contract.value("executionSpecificityScore", 0);
} else {
item.executionSpecificityScore = itemJson.value("execution_specificity_score", 0);
}
item.strictExecutionContract = strictExecutionContract;
if (itemJson.contains("capability_signals") && itemJson["capability_signals"].is_object()) {
const json& c = itemJson["capability_signals"];
item.capabilityCompileFailed = c.value("compile_failed", false);
item.capabilityTestsFailed = c.value("tests_failed", false);
item.capabilityToolchainMissing = c.value("toolchain_missing", false);
}
items.push_back(std::move(item)); items.push_back(std::move(item));
} }
@@ -73,15 +259,114 @@
auto report = TaskitemQualityAuditor::audit(items, workspace); auto report = TaskitemQualityAuditor::audit(items, workspace);
json results = json::array(); json results = json::array();
std::map<std::string, int> gapClassCounts;
std::map<std::string, int> recommendedToolCounts;
std::size_t resultIndex = 0;
for (const auto& r : report.results) { for (const auto& r : report.results) {
std::string dominantGapClass = "none";
std::vector<std::string> lockIssues;
if (resultIndex < args["taskitems"].size()) {
const auto& itemJson = args["taskitems"][resultIndex];
if (itemJson.is_object() && itemJson.contains("resource_locks") && itemJson["resource_locks"].is_array()) {
bool hasGlobal = false;
bool hasLocal = false;
for (const auto& lock : itemJson["resource_locks"]) {
if (!lock.is_string()) {
lockIssues.push_back("resource_locks contains non-string entries");
continue;
}
const std::string lockName = lock.get<std::string>();
if (!resourceLockLooksValid(lockName)) {
lockIssues.push_back("resource lock has invalid format: " + lockName);
}
if (lockName.rfind("global:", 0) == 0) hasGlobal = true;
else hasLocal = true;
}
if (hasGlobal && hasLocal) {
lockIssues.push_back("resource lock scope ambiguity: mixed local and global locks");
}
}
}
if (!r.issues.empty()) {
bool hasCapabilityIssue = false;
for (const auto& issue : r.issues) {
if (issue.find("capability_signals indicate") != std::string::npos) {
hasCapabilityIssue = true;
break;
}
}
if (hasCapabilityIssue) {
dominantGapClass = "generator_capability_gap";
} else {
dominantGapClass = classifyValidationIssueGapClass(r.issues.front());
}
}
if (!lockIssues.empty()) dominantGapClass = "resource_lock_semantics_gap";
if (resultIndex < args["taskitems"].size()) {
const auto& itemJson = args["taskitems"][resultIndex];
if (itemJson.is_object()) {
json contract = json::object();
if (itemJson.contains("execution_contract") && itemJson["execution_contract"].is_object()) {
contract = itemJson["execution_contract"];
} else if (itemJson.contains("executionContract") && itemJson["executionContract"].is_object()) {
contract = itemJson["executionContract"];
}
const json targetFiles = contract.value("targetFiles", json::array());
if (hasCrossProjectTargetMismatch(targetFiles, expectedProjectRoots)) {
lockIssues.push_back("cross_project_target_mismatch: generated targetFiles do not match expected project roots");
dominantGapClass = "cross_project_targeting_gap";
}
}
}
json recommendedTools = recommendedToolsForValidationGap(dominantGapClass);
gapClassCounts[dominantGapClass]++;
if (recommendedTools.is_array()) {
for (const auto& toolName : recommendedTools) {
if (toolName.is_string()) recommendedToolCounts[toolName.get<std::string>()]++;
}
}
json issuesOut = json::array();
for (const auto& issue : r.issues) issuesOut.push_back(issue);
for (const auto& issue : lockIssues) issuesOut.push_back(issue);
results.push_back({ results.push_back({
{"task_id", r.taskId}, {"task_id", r.taskId},
{"score", r.score}, {"score", r.score},
{"execution_specificity_score", r.executionSpecificityScore},
{"self_contained", r.selfContained}, {"self_contained", r.selfContained},
{"issues", r.issues} {"issues", issuesOut},
{"gap_class", dominantGapClass},
{"recommended_tools", recommendedTools}
}); });
++resultIndex;
} }
json gapCountsJson = json::object();
for (const auto& kv : gapClassCounts) gapCountsJson[kv.first] = kv.second;
std::vector<std::pair<std::string, int>> rankedTools(recommendedToolCounts.begin(), recommendedToolCounts.end());
std::sort(rankedTools.begin(), rankedTools.end(), [](const auto& a, const auto& b) {
if (a.second != b.second) return a.second > b.second;
return a.first < b.first;
});
json topTools = json::array();
for (const auto& kv : rankedTools) {
topTools.push_back({{"tool", kv.first}, {"count", kv.second}});
}
const bool executionPass = report.failingCount == 0;
const bool semanticPass = executionPass && report.warningCount == 0;
const bool recursivePass = executionPass && report.averageExecutionSpecificityScore >= 60.0;
const double confidence = std::max(0.0, std::min(1.0, (report.averageScore / 100.0) * (report.averageExecutionSpecificityScore / 100.0)));
bool hasCapabilityGap = false;
for (const auto& kv : gapClassCounts) {
if (kv.first == "generator_capability_gap" && kv.second > 0) {
hasCapabilityGap = true;
break;
}
}
json capabilityGapCallPlan = buildCapabilityGapCallPlan(hasCapabilityGap, strictExecutionContract);
return { return {
{"success", true}, {"success", true},
{"report", { {"report", {
@@ -90,9 +375,18 @@
{"warning_count", report.warningCount}, {"warning_count", report.warningCount},
{"failing_count", report.failingCount}, {"failing_count", report.failingCount},
{"average_score", report.averageScore}, {"average_score", report.averageScore},
{"average_execution_specificity_score", report.averageExecutionSpecificityScore},
{"results", results}, {"results", results},
{"top_issues", report.topIssues} {"top_issues", report.topIssues},
{"gap_class_counts", gapCountsJson},
{"top_recommended_tools", topTools},
{"capability_gap_call_plan", capabilityGapCallPlan},
{"promotion_packet", {
{"execution_pass", executionPass},
{"semantic_pass", semanticPass},
{"recursive_pass", recursivePass},
{"confidence", confidence}
}}
}} }}
}; };
} }

View File

@@ -1,4 +1,4 @@
// Step 615: whetstone_generate_taskitems MCP Tool (12 tests) // Step 615: whetstone_generate_taskitems MCP Tool (13 tests)
#include "MCPServer.h" #include "MCPServer.h"
@@ -199,6 +199,37 @@ void test_rejects_invalid_requirement_kind() {
PASS(); PASS();
} }
void test_cross_project_repo_signal_grounds_target_files() {
TEST(cross_project_repo_signal_grounds_target_files);
MCPServer mcp;
json reqs = json::array({{
{"requirementId", "goal-constcad"},
{"kind", "goal"},
{"normalizedText", "ConstCAD ecosystem integration across cad_orchestrator, hivemind, gleaner, and constcad"},
{"anchor", "goals"},
{"sourceLine", 1},
{"ambiguous", false}
}});
json out = callTool(mcp, "whetstone_generate_taskitems",
{{"normalizedRequirements", reqs}, {"strictExecutionContract", true}});
CHECK(out.value("success", false), "success should be true");
const auto& files = out["tasks"][0]["executionContract"]["targetFiles"];
bool hasExternal = false;
for (const auto& f : files) {
if (!f.is_string()) continue;
std::string v = f.get<std::string>();
if (v.rfind("cad_orchestrator/", 0) == 0 ||
v.rfind("hivemind/", 0) == 0 ||
v.rfind("gleaner/", 0) == 0 ||
v.rfind("constcad/", 0) == 0) {
hasExternal = true;
}
CHECK(v.rfind("editor/src/mcp/", 0) != 0, "should not fallback to editor/src/mcp internal defaults");
}
CHECK(hasExternal, "expected at least one cross-project target file");
PASS();
}
int main() { int main() {
std::cout << "Step 615: whetstone_generate_taskitems MCP Tool\n"; std::cout << "Step 615: whetstone_generate_taskitems MCP Tool\n";
@@ -214,6 +245,7 @@ int main() {
test_normalized_requirements_must_be_array(); // 10 test_normalized_requirements_must_be_array(); // 10
test_rejects_empty_normalized_requirements(); // 11 test_rejects_empty_normalized_requirements(); // 11
test_rejects_invalid_requirement_kind(); // 12 test_rejects_invalid_requirement_kind(); // 12
test_cross_project_repo_signal_grounds_target_files(); // 13
std::cout << "\nResults: " << passed << "/" << (passed + failed) << " passed\n"; std::cout << "\nResults: " << passed << "/" << (passed + failed) << " passed\n";
return failed == 0 ? 0 : 1; return failed == 0 ? 0 : 1;

View File

@@ -1,4 +1,4 @@
// Step 682: whetstone_validate_taskitem MCP tool (8 tests) // Step 682: whetstone_validate_taskitem MCP tool (9 tests)
#include "MCPServer.h" #include "MCPServer.h"
@@ -141,14 +141,34 @@ void t8() {
TEST(tool_count_updated_to_88); TEST(tool_count_updated_to_88);
auto d = loadToolsJson(); auto d = loadToolsJson();
CHECK(d.contains("tools"), "tools missing"); CHECK(d.contains("tools"), "tools missing");
CHECK(d["tools"].size() == 88, "expected 88 tools"); CHECK(d["tools"].size() >= 88, "expected at least 88 tools");
PASS();
}
void t9() {
TEST(cross_project_target_mismatch_classified);
MCPServer mcp;
json item = validTaskitem();
item["execution_contract"] = {
{"targetFiles", json::array({
"editor/src/mcp/RegisterArchitectIntakeTools.h",
"tools/mcp/run_sprint_taskitem_pipeline.sh"
})}
};
auto out = callTool(mcp, "whetstone_validate_taskitem", {
{"taskitems", json::array({item})},
{"expected_project_roots", json::array({"cad_orchestrator", "hivemind", "gleaner", "constcad"})}
});
CHECK(out.value("success", false), "expected success");
CHECK(out["report"]["results"].size() == 1, "expected one result");
CHECK(out["report"]["results"][0].value("gap_class", "") == "cross_project_targeting_gap",
"expected cross_project_targeting_gap");
PASS(); PASS();
} }
int main() { int main() {
std::cout << "Step 682: whetstone_validate_taskitem MCP tool\n"; std::cout << "Step 682: whetstone_validate_taskitem MCP tool\n";
t1(); t2(); t3(); t4(); t5(); t6(); t7(); t8(); t1(); t2(); t3(); t4(); t5(); t6(); t7(); t8(); t9();
std::cout << "\nResults: " << passed << "/" << (passed + failed) << " passed\n"; std::cout << "\nResults: " << passed << "/" << (passed + failed) << " passed\n";
return failed == 0 ? 0 : 1; return failed == 0 ? 0 : 1;
} }

View File

@@ -336,6 +336,15 @@ if [[ "$SEMANTIC_PLANNING_BRIDGE" == "1" && "$SEMANTIC_REQUIREMENT_INJECTION" ==
SEM_REQS="$(jq -nc --argjson packet "$(cat "$OUT_DIR/00b_semantic_planning_annotations.json")" ' SEM_REQS="$(jq -nc --argjson packet "$(cat "$OUT_DIR/00b_semantic_planning_annotations.json")" '
def mk($kind; $text; $id): def mk($kind; $text; $id):
{requirementId:$id, kind:$kind, normalizedText:$text, anchor:"semantic_packet", sourceLine:0, ambiguous:false}; {requirementId:$id, kind:$kind, normalizedText:$text, anchor:"semantic_packet", sourceLine:0, ambiguous:false};
def backticks($s):
($s | split("`")) as $parts
| [ range(1; ($parts|length); 2) as $i | $parts[$i] ];
def normalize_path($p):
($p | gsub("^\\s+|\\s+$"; "")) as $t
| if ($t|startswith("/home/bill/Documents/")) then ($t | sub("^/home/bill/Documents/"; ""))
else $t end;
def is_file_path($p):
($p | test("/") and test("\\.[A-Za-z0-9]{1,8}$") and (contains(" ")|not) and (contains("<")|not) and (contains(">")|not));
[ [
(if ($packet.annotations[]? | select(.type=="intent") | .fields.summary // "" | length) > 0 (if ($packet.annotations[]? | select(.type=="intent") | .fields.summary // "" | length) > 0
then mk("goal"; ($packet.annotations[] | select(.type=="intent") | .fields.summary); "semantic-intent-summary") then mk("goal"; ($packet.annotations[] | select(.type=="intent") | .fields.summary); "semantic-intent-summary")
@@ -347,8 +356,15 @@ if [[ "$SEMANTIC_PLANNING_BRIDGE" == "1" && "$SEMANTIC_REQUIREMENT_INJECTION" ==
(($packet.annotations[]? | select(.type=="contract") | .fields.constraints[]?) as $c (($packet.annotations[]? | select(.type=="contract") | .fields.constraints[]?) as $c
| mk("constraint"; ("semantic contract clause: " + $c); ("semantic-contract-" + ($c|gsub("[^A-Za-z0-9]+";"_"))))), | mk("constraint"; ("semantic contract clause: " + $c); ("semantic-contract-" + ($c|gsub("[^A-Za-z0-9]+";"_"))))),
(($packet.annotations[]? | select(.type=="capability_requirement") | .fields.capability?) as $cap (($packet.annotations[]? | select(.type=="capability_requirement") | .fields.capability?) as $cap
| mk("dependency"; ("semantic capability requirement: " + $cap); ("semantic-capability-" + ($cap|gsub("[^A-Za-z0-9]+";"_"))))) | mk("dependency"; ("semantic capability requirement: " + $cap); ("semantic-capability-" + ($cap|gsub("[^A-Za-z0-9]+";"_"))))),
] | map(select(.normalizedText | length > 0)) (($packet.annotations[]? | select(.type=="contract") | .fields.required_artifacts[]?) as $a
| backticks($a)[]?
| normalize_path(.)
| select(is_file_path(.))
| mk("requirement"; ("explicit target file: " + .); ("semantic-artifact-file-" + (. | gsub("[^A-Za-z0-9]+";"_")))))
]
| map(select(.normalizedText | length > 0))
| unique_by(.requirementId)
')" ')"
printf '%s\n' "$SEM_REQS" > "$OUT_DIR/01c_semantic_injected_requirements.json" printf '%s\n' "$SEM_REQS" > "$OUT_DIR/01c_semantic_injected_requirements.json"
NORMALIZED_REQS="$(jq -nc --argjson base "$NORMALIZED_REQS" --argjson sem "$SEM_REQS" '$base + $sem')" NORMALIZED_REQS="$(jq -nc --argjson base "$NORMALIZED_REQS" --argjson sem "$SEM_REQS" '$base + $sem')"