Training large models on a single GPU is limited by memory constraints. Distributed training enables scalable training across multiple GPUs.