bill/whetstone_DSL
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
792 Commits 2 Branches 0 Tags
001-core-ast-structure
Commit Graph

3 Commits

Author SHA1 Message Date
Bill
72ffee68fa WIP: stage all uncommitted work — sprints 46-221, graduation headers, specialist fleet, test steps 909-1988 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-22 10:15:48 -06:00
Bill
15125dce25 Specialist fleet: pilot sweep results + automatability decomposition 
Pilot sweep (all 4 gates, small_plus + medium):
  verification_type  84.3%  (medium, no_gain_stop_scaling)
  worker_type        80.8%  (small_plus, scale_more)
  automatability     78.3%  (medium, decompose_or_relabel)
  prereq_op          72.2%  (small_plus, decompose_or_relabel)

Automatability: 2-stage decomposition resolves Directional Collapse.
  Stage 1 binary (low_entropy vs needs_routing): 100%
  Stage 2 4-way (specialist/slm/llm/human): 100%
  Combined 5-way pipeline: 100%
  Script: specialists/scripts/split_automatability_decompose.py

Worker_type: combined dataset (sprint plans + projects) improves
  implementer class from 42.9% → 75.0%, overall 80.8% → 92.2%.
  Combined train/eval TSVs added.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-04-19 13:44:38 -06:00
Bill
3fc9deac5b specialists: switch training to PyTorch, add capacity sweep infrastructure 
- train_specialist_pt.py: pure PyTorch trainer replacing Fabricate binary.
  No SQLite/WAL — saves checkpoint.pt + history.json only. Fabricate was
  causing D-state IO blocking on the HDD due to continuous WAL writes.
- eval_specialist_pt.py: PyTorch eval with confusion matrix, threshold
  analysis, guardrail assessment, and deployment verdict.
- capacity_sweep.py: updated to call PyTorch scripts; TIERS now include
  heads field (4/6/8) since PyTorch supports multi-head unlike Fabricate.
- prereq_op_binary_split.py: evaluate_combined rewritten in PyTorch.
- Performance fix in train_specialist_pt.py: load training data onto GPU
  once and sample via torch.randint instead of DataLoader (eliminates
  Python/CPU overhead on tiny datasets). Fix applied, not yet timed.

Next session: time the fix, then run sweep_all_gates.sh --pilot-only.
See HANDOFF-2026-03-31.md for full context.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-31 22:55:51 -06:00