Runbook: Fine-Tuning (Phase 5)¶

Building datasets, launching training, evaluating, and managing the registry.

Prerequisites¶

pip install -e ".\backend[finetune]"   # peft, trl, datasets, accelerate, eval metrics
docker compose up -d redis             # job progress bus
ollama serve                            # only for RAGAS-style eval

Hardware: CPU-only here. sft_lora and dpo train on a tiny model; qlora/rlhf require a CUDA GPU and fail with a clear error otherwise.

Build a dataset¶

# from a JSONL of {prompt, completion} (SFT) or {prompt, chosen, rejected} (preference)
aop finetune dataset mydata --jsonl records.jsonl --format sft

Train (streams live progress)¶

aop finetune train --dataset data/datasets/mydata/train.jsonl --name myagent `
                   --method sft_lora --base-model sshleifer/tiny-gpt2 --max-steps 20
aop finetune jobs
aop finetune registry

Training runs in an isolated subprocess (PYTHONUTF8=1) and publishes progress to Redis; the CLI/UI tail it.

Evaluate¶

# samples.jsonl rows: {prediction, reference} and/or {question, answer, context}
aop finetune eval samples.jsonl --metrics rouge,bertscore,ragas

UI¶

aop serve then open the canvas → Fine-Tuning → (or /finetune): launch form, live progress, registry/jobs tables.

Troubleshooting¶

Symptom	Cause	Fix
`HardwareUnavailable: requires a CUDA GPU`	qlora/rlhf on CPU	use `sft_lora`/`dpo`, or run on a GPU host.
Windows: `'charmap' codec` on import	TRL template encoding	training already runs under `PYTHONUTF8=1`; set it if importing TRL directly.
job never streams	Redis down	`docker compose up -d redis`.
`ModuleNotFoundError: peft/trl`	extra not installed	`pip install -e ".[finetune]"`.
slow / OOM	base model too big for CPU	use a tiny model + few steps, or a GPU.

Tuning¶

AOP_FINETUNE_BASE_MODEL, AOP_FINETUNE_MAX_STEPS, AOP_FINETUNE_REGISTRY_DIR, AOP_FINETUNE_DATASET_DIR, AOP_FINETUNE_EVAL_BERT_MODEL.