whetstone_DSL/sprint115_plan.md

# Sprint 115 Plan: Requirements Derivation from AST

## Context

Environment setup is currently manual and implicit. A developer knows their
project needs Python 3.11, numpy, and a running PostgreSQL instance because
they remember building it — not because the system can read that from the code.

Sprint 115 makes requirements derivation deterministic: given an AST, walk all
import/include/dependency nodes and produce a structured `RequirementsManifest`
— a language-agnostic list of what the project needs to run. From that manifest,
generate an idempotent setup script in the appropriate package manager format
(apt, pip, npm, cargo, cmake/vcpkg). Cross-reference the manifest against an
`EnvironmentSnapshot` (Sprint 112) to produce a gap report: what's missing,
what's the wrong version, what's ready.

This is the "environment as code" step: the project's needs become a
machine-readable artifact that any agent can check, compare, and act on.

---

## Goals

1. Define `RequirementsManifest` schema derived from the AST
2. Walk all dependency nodes across supported languages
3. Emit setup scripts per package manager (apt, pip, npm, cargo, vcpkg)
4. Cross-reference manifest against an EnvironmentSnapshot to get a gap report
5. Expose derivation and gap report as MCP tools

---

## Steps

### Step 1379: `RequirementsManifest` schema (12 tests)

Create `editor/src/RequirementsManifest.h`.

Fields: `manifest_id`, `source_file` (optional — if derived from single file),
`workspace` (optional — if derived from full project), `language_runtimes[]`
(name, min_version, source), `packages[]` (name, version_constraint,
package_manager: apt/pip/npm/cargo/vcpkg/brew/unknown),
`system_services[]` (name, port — e.g. postgres:5432, redis:6379),
`env_vars_required[]` (key, description, has_default: bool),
`derived_at`, `schema_version`.

Tests (12): constructable with defaults, manifest_id non-empty, packages
list starts empty, runtimes list starts empty, services list starts empty,
env_vars list starts empty, JSON serialization, JSON deserialization,
package_manager enum round-trips, unknown manager passthrough, schema_version
present, deterministic output.

### Step 1380: Import/include dependency walker (10 tests)

Create `editor/src/DependencyWalker.h`.

Walks an open AST, finds all import/include/require nodes, and extracts
package names. Language-specific strategies:
- Python: `import X`, `from X import Y` → package name is top-level module
- C/C++: `#include <X>` → system header; `#include "X"` → local (skip)
- JavaScript/TypeScript: `require('X')`, `import ... from 'X'` → npm package
- Rust: `extern crate X`, `use X::` → cargo crate
- Java: `import X.Y.Z` → Maven group

Stdlib modules filtered via a known-stdlib list per language (extensible).

Tests (10): Python import extracted, Python stdlib filtered, C++ system header
extracted, C++ local include skipped, JS require extracted, TS import extracted,
Rust extern crate extracted, Java import extracted, batch walk across multiple
files, deterministic output order.

### Step 1381: Runtime version inferrer (10 tests)

Create `editor/src/RuntimeVersionInferrer.h`.

Infers language runtime requirements from project configuration files:
- Python: `pyproject.toml` `requires-python`, `.python-version`, `setup.py`
- Node.js: `package.json` `engines.node`, `.nvmrc`
- Rust: `rust-toolchain.toml`
- C++: CMake `CMAKE_CXX_STANDARD`
- Java: `pom.xml` `maven.compiler.source`

Falls back to "any recent version" if no constraint found (min_version = "").

Tests (10): pyproject.toml version read, .python-version read, package.json
engines read, .nvmrc read, rust-toolchain.toml read, cmake standard read,
pom.xml source read, missing file returns no-constraint, multiple constraints
take strictest, deterministic output.

### Step 1382: Service dependency detector (10 tests)

Create `editor/src/ServiceDependencyDetector.h`.

Detects service dependencies from config files and connection string patterns
in source. Scans `.env`, `config.yaml`, `appsettings.json` for patterns like
`POSTGRES_HOST`, `REDIS_URL`, `MONGO_URI`, `DATABASE_URL`. Maps to
`SystemServiceRecord` (name, port).

Also detects from source: `psycopg2.connect(...)`, `mongoose.connect(...)`,
`redis.Redis(...)` call patterns in the AST.

Tests (10): postgres from .env POSTGRES_HOST, redis from REDIS_URL, mongo
from MONGO_URI, mysql from DATABASE_URL pattern, psycopg2 call in AST,
mongoose call in AST, redis.Redis call in AST, no false positives on
unrelated config, deterministic output, unknown service captured as unknown.

### Step 1383: Setup script generator (10 tests)

Create `editor/src/SetupScriptGenerator.h`.

Given a `RequirementsManifest`, generates an idempotent shell script per
requested package manager. Scripts use guard patterns (apt: check dpkg,
pip: check importlib, npm: check node_modules, cargo: check cargo metadata).

Output: `SetupScript` with fields `package_manager`, `script_text`,
`estimated_packages` (count), `requires_sudo` (bool).

One `whetstone_generate_setup_script` call may return multiple scripts
(one per relevant package manager detected from the manifest).

Tests (10): apt script generated for apt packages, pip script generated,
npm script generated, cargo script generated, idempotent guard present in
all scripts, sudo flag set for apt, not set for pip/npm/cargo, empty package
list produces no-op script, deterministic output, script text is valid shell.

### Step 1384: Requirements gap report (8 tests)

Create `editor/src/RequirementsGapReport.h`.

Cross-references `RequirementsManifest` against `EnvironmentSnapshot`.

Fields: `ready` (bool), `satisfied[]` (package name),
`missing[]` (name + required_version + package_manager),
`version_mismatch[]` (name + required + installed),
`unverifiable[]` (packages where installed version can't be confirmed),
`service_gaps[]` (services in manifest not found in snapshot processes).

Tests (8): ready true when all satisfied, missing package detected, version
mismatch detected, correct version passes, service gap detected, unverifiable
when probe can't confirm, JSON round-trip, deterministic output.

### Step 1385: `whetstone_derive_requirements` MCP tool (8 tests)

Input: `{ "workspace": string?, "files": string[]? }`
Output: `RequirementsManifest` JSON.

Walks all open buffers (or specified files), runs DependencyWalker +
RuntimeVersionInferrer + ServiceDependencyDetector, merges results.

Tests (8): tool registered, packages field present, runtimes field present,
services field present, empty workspace returns empty manifest, single file
mode works, manifest_id returned, deterministic output.

### Step 1386: `whetstone_generate_setup_script` MCP tool (8 tests)

Input: `{ "manifest_id": string?, "manifest": RequirementsManifest? }`
Output: `{ "scripts": SetupScript[] }`

Accepts either a previously derived manifest by ID or an inline manifest.

Tests (8): tool registered, scripts returned, apt script present for C++
project, pip script present for Python project, idempotent guards in output,
empty manifest returns empty scripts, manifest_id lookup works, deterministic.

### Step 1387: `whetstone_verify_requirements` MCP tool (8 tests)

Input: `{ "manifest_id": string, "snapshot_id": string }`
Output: `RequirementsGapReport` JSON.

Loads both from store, computes gap report.

Tests (8): tool registered, ready true when all installed, missing reported,
mismatch reported, service gap reported, missing manifest_id error, missing
snapshot_id error, deterministic output.

### Step 1388: Sprint 115 integration summary + regression (8 tests)

Create `editor/src/Sprint115IntegrationSummary.h`.

End-to-end: open Python project → derive_requirements → snapshot_environment →
verify_requirements → generate_setup_script → verify gap report matches missing
packages in snapshot.

Tests (8): headers constructable, tools registered, end-to-end flow works,
gap report accurate, setup script produced, no header exceeds 600 lines,
deterministic snapshots, full prior regression passes.

---

## Architecture Gate

- Dependency walker uses AST node traversal only — no regex on raw source text
- Service detection from config files uses ConfigFileProbe (Sprint 112) —
  no duplicated parsing logic
- Setup scripts are never executed by Whetstone — generation only, execution
  is the caller's responsibility
- Max 600 lines per header