whetstone_DSL/docs/lora_qwen14b_training_runbook.md

Qwen2.5-Coder-14B LoRA Runbook (MCP Quality)

Objective

Train a first LoRA that improves MCP tool-call quality judgments and recovery behavior using the prepared grouped splits.

Inputs

• training_data/lora/splits/mcp_lora.train.jsonl
• training_data/lora/splits/mcp_lora.val.jsonl
• training_data/lora/splits/mcp_lora.test.jsonl

Locked config

• Config file: configs/lora/qwen2.5-coder-14b-quality.env
• Launcher: tools/mcp/train_qwen14b_quality_lora.sh

Preflight

cd /home/bill/Documents/CLionProjects/whetstone_DSL
./tools/mcp/check_lora_training_readiness.sh

Python dependencies (one-time)

python3 -m pip install --upgrade pip
python3 -m pip install torch transformers peft trl bitsandbytes datasets accelerate

Build SFT chat dataset

python3 ./tools/mcp/build_lora_quality_sft_dataset.py \
  --train-in training_data/lora/splits/mcp_lora.train.jsonl \
  --val-in training_data/lora/splits/mcp_lora.val.jsonl \
  --test-in training_data/lora/splits/mcp_lora.test.jsonl \
  --out-dir training_data/lora/sft_qwen14b_quality

Train

./tools/mcp/train_qwen14b_quality_lora.sh

Pass/Fail gates for first checkpoint

• json_parse_success >= 99.5%
• invalid_schema_rate <= 1.0%
• label_f1 >= 0.92
• failure_class_macro_f1 >= 0.80
• tool_selection_error_rate <= 5%

Notes

• Keep this LoRA scoped to qwen2.5-coder:14b data. Do not merge with other base-model families in the same adapter.
• Use grouped splits to avoid run-family leakage between train/val/test.