Training

Overview

Training updates model parameters (weights) via backpropagation to minimize a loss function over a dataset. For LLMs, this spans three phases: pre-training on trillions of tokens, fine-tuning on curated data, and alignment training (RLHF/DPO/CAI) to align with human preferences.

Cost and scale

Pre-training a GPT-4 class frontier model costs tens to hundreds of millions of dollars in compute. Parameter-Efficient Fine-Tuning methods (LoRA, QLoRA), however, are feasible on consumer hardware, making domain-specific model creation practical for SMBs.