88 lines
3.0 KiB
Markdown
88 lines
3.0 KiB
Markdown
# Multi-Model LoRA Data Playbook
|
|
|
|
## Goal
|
|
Build a reusable dataset where failures come from multiple models and good examples come from validated pipeline outputs, then rebalance into a training-ready JSONL.
|
|
|
|
## Required metadata per record
|
|
Every record should carry:
|
|
- `model` (for example `qwen2.5-coder:14b`, `qwen2.5-coder:7b`)
|
|
- `label` (`good` or `bad`)
|
|
- `interface_mode` (`json_loop` or `native_tool_calling`)
|
|
- `source` (for example `pipeline_run`, `stage_labeled`, `mcp_tool_response`)
|
|
|
|
## Collection flow
|
|
1. Run collection loops per model.
|
|
2. Export malformed and quality datasets after each run.
|
|
3. Keep run manifests for token/cycle diagnostics.
|
|
4. Build one balanced multimodel output before training.
|
|
|
|
## Suggested run strategy
|
|
- Keep one primary target model (for example 14B) as highest share of data.
|
|
- Add 1-2 secondary models for failure diversity.
|
|
- Avoid overfitting to one repeated failure signature.
|
|
|
|
## Commands
|
|
|
|
### 0) Good-data hybrid flow (deterministic + optional stronger-LLM review)
|
|
```bash
|
|
# Step A: deterministic good-data growth from run specs (same corpora as bad-data loop by default)
|
|
./tools/mcp/generate_good_from_run_specs.sh
|
|
|
|
# Step B: split good records into high-confidence and review queue
|
|
./tools/mcp/build_good_hybrid_dataset.sh
|
|
|
|
# One-command orchestrator (A+B)
|
|
./tools/mcp/run_good_data_hybrid.sh
|
|
|
|
# After stronger-LLM review creates approved-good JSONL:
|
|
REVIEW_APPROVED_FILE=training_data/lora/mcp_calls_good_review_approved.jsonl \
|
|
./tools/mcp/run_good_data_hybrid.sh
|
|
```
|
|
|
|
### 1) Per-run exports
|
|
```bash
|
|
./tools/mcp/export_malformed_mcp_calls_for_lora.sh logs/taskitem_runs/<bridge_run_dir>
|
|
./tools/mcp/export_mcp_call_quality_for_lora.sh logs/taskitem_runs/<good_pipeline_run_dir> logs/taskitem_runs/<bridge_run_dir>
|
|
```
|
|
|
|
### 2) Build balanced multimodel dataset
|
|
```bash
|
|
BAD_TO_GOOD_RATIO=1.0 \
|
|
MAX_PER_MODEL_PER_LABEL=250000 \
|
|
REQUIRE_MODEL_METADATA=0 \
|
|
DEFAULT_MODEL_TAG=unknown_model \
|
|
DEFAULT_INTERFACE_MODE=json_loop \
|
|
./tools/mcp/build_multimodel_lora_dataset.sh
|
|
```
|
|
|
|
Outputs:
|
|
- `training_data/lora/mcp_multimodel_balanced.jsonl`
|
|
- `training_data/lora/mcp_multimodel_balanced.summary.json`
|
|
|
|
### 3) Create leakage-safe train/val/test split (grouped by run/spec family)
|
|
```bash
|
|
./tools/mcp/split_lora_dataset_by_group.sh
|
|
```
|
|
|
|
Outputs:
|
|
- `training_data/lora/splits/mcp_lora.train.jsonl`
|
|
- `training_data/lora/splits/mcp_lora.val.jsonl`
|
|
- `training_data/lora/splits/mcp_lora.test.jsonl`
|
|
- `training_data/lora/splits/mcp_lora.split_summary.json`
|
|
|
|
## Quality gates before training
|
|
- Confirm `selected_good` and `selected_bad` counts are both non-trivial.
|
|
- Confirm multiple models appear in `by_model_selected`.
|
|
- Confirm source diversity in `by_source_selected`.
|
|
- Spot-check for infra-only failures; keep operational noise low.
|
|
|
|
## For future agents
|
|
When collecting new runs:
|
|
- Do not overwrite raw files.
|
|
- Rebuild balanced output from raw files each time.
|
|
- Record all run IDs and model tags in commit notes.
|
|
|
|
When training:
|
|
- Train separate LoRAs per base model family/size.
|
|
- Reuse the same balanced dataset recipe; do not reuse adapters across base sizes.
|