Quantization: fewer bits per number
Lower-precision representations can reduce model storage and memory traffic while introducing approximation.
In this lesson
Quantization compresses numerical representations; it does not simply delete words from a model’s knowledge.
A small numerical vocabulary
A high-precision weight can take many representable values. Quantization maps it to a smaller set, often with a scale and sometimes an offset used to reconstruct an approximation. The network then computes with these approximations or with specialized operations that use the compressed representation.
For an original toy example, use a scale of 0.1 and round to the nearest integer. A weight 0.37 becomes integer 4 and reconstructs as 0.4; the error is +0.03. A weight −0.26 becomes −3 and reconstructs as −0.3; the error is −0.04. A real method must also define representable bounds and what happens outside them.
Weights are only one target
Weight-only quantization compresses parameter storage. Other approaches also quantize activations or caches. These choices affect different parts of memory use and computation. The bit width alone does not specify a method: grouping, scales, calibration data, outlier handling, and supported hardware all matter.
GPTQ, introduced in 2022, is a post-training weight-quantization approach that uses approximate second-order information. AWQ, introduced in 2023, uses activation information to guide weight quantization. Their experimental results show particular tradeoffs; they do not establish one universal quality loss for all four-bit models.
Why the approximation can matter
Changing a small weight can alter later activations, logits, and ultimately a sampled token. Many small numerical changes may have little visible effect on one prompt and matter on another. A model that performs similarly on average can still change on a sensitive task or long input. Evaluate the actual quantized artifact.
Calibration data help some methods decide where precision matters. If those examples poorly represent the deployment workload, the measured tradeoff can be misleading. The compression procedure also needs to match the intended runtime; a file that is smaller on disk is not automatically faster on hardware without efficient kernels for its format.
Calculate the benefit honestly
Raw weight storage is approximately parameter count multiplied by bits per parameter, divided by eight. A hypothetical one-billion-parameter array uses about 2 GB at 16 bits or 0.5 GB at 4 bits, before scales, metadata, alignment, and runtime overhead. This arithmetic is a lower-level estimate, not a complete device-memory requirement.
Quantization is useful because memory and bandwidth are real constraints. The correct decision balances quality, memory, speed, compatibility, and the task’s tolerance for changed behavior. Keep the original and quantized model versions identifiable so evaluation results remain reproducible and a compression regression can be traced.
Check the rounding error
With step size 0.1 and nearest rounding, an in-range number has absolute rounding error at most 0.05. Clipping an out-of-range value can exceed that bound. Always state the range as well as the step size.
A four-bit model always uses exactly one quarter of the total memory of a sixteen-bit model.
Only the raw weight term has that simple ratio. Scales, caches, activations, and runtime buffers remain.
Take it with you
- Bit width describes precision, not total system quality.
- Distinguish weight, activation, and cache quantization.
- Measure the resulting artifact on representative tasks.
Check your understanding.
Choose an answer, then see the explanation. You can try again.
Practice only. Your answer is not stored or sent anywhere.
Context management: choosing what stays in view
Continue when you’re ready. Marking this lesson as read is optional.
Sources & further reading
Primary references for this lesson. The explanation and examples above are original.
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers ↗Elias Frantar et al. · 2022-10-31 · checked 13 Nov 2026
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration ↗Ji Lin et al. · 2023-06-01; accessed revision 2026-04-25 · checked 13 Nov 2026