Bill
|
15125dce25
|
Specialist fleet: pilot sweep results + automatability decomposition
Pilot sweep (all 4 gates, small_plus + medium):
verification_type 84.3% (medium, no_gain_stop_scaling)
worker_type 80.8% (small_plus, scale_more)
automatability 78.3% (medium, decompose_or_relabel)
prereq_op 72.2% (small_plus, decompose_or_relabel)
Automatability: 2-stage decomposition resolves Directional Collapse.
Stage 1 binary (low_entropy vs needs_routing): 100%
Stage 2 4-way (specialist/slm/llm/human): 100%
Combined 5-way pipeline: 100%
Script: specialists/scripts/split_automatability_decompose.py
Worker_type: combined dataset (sprint plans + projects) improves
implementer class from 42.9% → 75.0%, overall 80.8% → 92.2%.
Combined train/eval TSVs added.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
2026-04-19 13:44:38 -06:00 |
|