Guide 41 · LLM Training, Fine-Tuning and Serving

DeepSpeed and Megatron-LM, Deep Dive

Splitting a model and its optimizer across a cluster, and the memory arithmetic that decides the settings.

Planned outline not written yet

Before this guide: Accelerate, Deep Dive.

The notebooks

The outline for this guide is not written yet. It will name every notebook, in reading order, before the first one is written.