ucwm/docs/roadmap.md

UCWM Roadmap

Goal

A working, inspectable, contract-tested language reasoning system that converts natural language into structured world state, resolves constraints, and produces answers from that structure — not from token pattern matching.

Guiding principle

Build the thinnest possible vertical slice first. Prove the pipeline shape. Replace rule-based components with trained models only after contracts are stable and the slice works end-to-end.

---

Phase 1 — Vertical Slice

Goal: One input in, one correct answer out, with a full inspectable trace at every stage.
Target input: "Alice had 5 apples. She gave Bob 2 apples. How many apples does Alice have now?"
Target output: "Alice has 3 apples." with derivation trace.

Sprint	Name	Goal
1	Skeleton	C++ project compiles. All core types defined. WorldState round-trips to/from JSON.
2	Gates + Proposal	5 keyword gates fire correctly. Object proposal extracts entities, quantities, events from apple problem.
3	Specialists	Entity specialist resolves Alice/She. Quantity specialist emits arithmetic constraint and resolves it.
4	Resolver + Synthesis + Pipeline	Full pipeline runs. Apple problem produces correct answer with debug trace. All contract tests pass.

Phase 1 exit criteria:

• Apple problem works end-to-end
• Each layer has passing contract tests
• Debug trace shows full derivation
• A failure in one layer is diagnosable without inspecting others

---

Phase 2 — Domain Expansion

Goal: Expand to more input types. Stress-test contracts. Add the specialists the apple problem skipped.

Sprint	Name	Goal
5	Ownership Specialist	Track possession, transfers, state at time T. Multi-step ownership chains.
6	Temporal Specialist	Before/after/during. Temporal chain example (temporal_001.json) works end-to-end.
7	Causal Specialist	Cause/effect/enabling/blocking. First causal example works.
8	Logic Specialist	Negation, implication, contradiction detection.
9	Multi-step Problems	3+ step word problems. Resolver handles chains of arithmetic and ownership constraints.
10	Robustness Sprint	Edge cases: empty input, ambiguous entities, contradicting constraints. Error states propagate correctly.

Phase 2 exit criteria:

• All 5 core specialist domains covered (entity, quantity, temporal, causal, logic)
• At least 10 distinct example fixtures with passing end-to-end tests
• open_contradictions and error states tested explicitly
• Decision matrix completed for gate implementation (regex vs. transformer)

---

Phase 3 — Neural Gates

Goal: Replace keyword gates with a trained multi-label transformer classifier. Gate contract unchanged.

Sprint	Name	Goal
11	Training Data	Generate labeled gate training examples (synthetic + manual). At least 500 examples per gate.
12	Gate Model	Fine-tune small encoder (DistilBERT-scale) with 17 sigmoid heads on labeled data.
13	ONNX Integration	Export gate model to ONNX. Load and run via ONNX Runtime in C++. Pass gate contract tests.
14	Calibration	Confidence scores calibrated against held-out set. Recall >= 0.90 on each gate.

Phase 3 exit criteria:

• Gate model passes all contract tests from tests/contracts/gate_contract_tests.md
• Recall >= 0.90 per gate
• Latency per gate array pass <= 20ms on CPU
• Keyword gates retained as fallback (togglable)
• All downstream behavior unchanged (contract held)

---

Phase 4 — Neural Specialists

Goal: Replace rule-based specialists with trained neural modules, one at a time.

Sprint	Name	Goal
15	Entity Specialist (Neural)	Small NER + coreference model. Passes entity contract tests. Replaces rule-based version.
16	Quantity Specialist (Neural)	Numeric extraction + arithmetic constraint emission. Handles implicit quantities.
17	Temporal Specialist (Neural)	Temporal ordering under ambiguity. Handles implicit sequence without explicit connectives.
18	Resolver Upgrade	Add probabilistic constraint handling. Soft constraint weighting. Multiple hypotheses.
19	Synthesis Upgrade	Move from template-based to generative synthesis. Answer quality evaluation.

Phase 4 exit criteria:

• Each specialist passes its contract tests with the neural implementation
• Performance equal to or better than rule-based on held-out examples
• Rule-based versions retained as reference implementations for debugging

---

Phase 5 — Perception Model

Goal: Build or integrate the broad ~1B parameter perception model that handles object proposal for novel domains.

Sprint	Name	Goal
20	Architecture Decision	Decision matrix: train from scratch vs. fine-tune existing. Encoder-only vs. encoder-decoder.
21	Perception Prototype	Small prototype perception model producing CanonicalObject proposals on unseen inputs.
22	Shared Encoder	Evaluate sharing the perception encoder with the gate classifier heads.
23	Integration	Wire perception model into pipeline. Replace rule-based object proposal.

---

Phase 6 — Scale and Specialized Domains

Goal: Expand to code, planning, social reasoning, and multi-turn conversation.

Sprint	Name	Goal
24	Code Specialist	AST-constrained generation. Code object proposals. Dependency graph constraints.
25	Planning Specialist	Step/goal/prerequisite tracking. Multi-step planning problems.
26	Social Specialist	Belief, intention, obligation, deception. Theory of mind examples.
27	Multi-turn	WorldState carries over across turns. Coreference across sentences.
28	Performance	Parallel specialist execution. Profiling. Latency targets.

---

Key decision points

These are forks in the road that should be decided with a decision matrix (see docs/decision_matrices.md) rather than ad hoc:

Decision	When
Gate implementation: keyword vs. neural	End of Phase 2
Specialist implementation: rule vs. neural (per specialist)	After each rule-based specialist is validated
Perception model: train from scratch vs. fine-tune	Start of Phase 5
Resolver: deterministic vs. probabilistic vs. SAT	End of Phase 3
Shared encoder: gate + perception vs. separate models	Start of Phase 5
Synthesis: template vs. generative	End of Phase 4

---

What this is not

• Not a chatbot. Multi-turn comes late (Sprint 27) because single-turn must be solid first.
• Not a 30B dense model. The goal is comparable usefulness via decomposition and structure.
• Not a research paper. The goal is a working system. Publish after it works.