Grain: retention versus synthesis
Film grain and sensor noise are often difficult to compress faithfully. Much of their fine variation changes from pixel to pixel and from frame to frame, so prediction helps little with it, though the scene under the grain stays predictable. An encoder can spend the bits to keep the grain, let quantization wipe it away, or take a third road: remove it before encoding and have the decoder put back grain that aims to resemble it without being the same.
Three ways to treat grain
- Retain it. Code the grainy picture as it is. At a fine quantizer the grain survives and is paid for. At a coarse one it is quantized away unevenly, which can leave flat patches and patches of leftover noise.
- Remove it. Denoise first. The picture codes cheaply and comes out clean, which may not be what its makers intended.
- Synthesize it. Denoise, code the clean picture, and send a few parameters describing the grain. The decoder generates new random grain to match and adds it to the picture it outputs.
Compare them
The cat has invented grain of the strength you set. This toy experiment knows the clean picture exactly, so its removal step is ideal: it cannot show a denoiser erasing texture by mistake. Coding is intra only, so nothing here measures grain over time. Error is the mean squared error of the output against the grainy source. Sky spread measures variation around each row's mean in the two top corners, where the picture without grain is plain sky. It mixes grain with coding artifacts and says nothing about whether the pattern matches. The Reconstructed panel is the picture rebuilt from what was coded; Output adds synthesized grain to it when the method calls for that. Nonzero levels counts the picture's coefficients only, not grain parameters or other syntax, so the equal counts of Remove and Synthesize do not mean equal streams or equal decoding work.
| Method | Nonzero levels | Error | Sky spread |
|---|---|---|---|
| Retain | |||
| Remove | |||
| Synthesize |
At grain 8 and QP 28:
- Retaining the grain took 413 nonzero levels; the other two took 367. With QP held at 28, grain 12 makes it 502 against 367; with grain held at 8, QP 22 makes it 664 against 477. In this experiment stronger grain and a finer quantizer each meant more nonzero levels to keep the grain.
- Synthesis brought the sky spread close to the source's and had the highest error of the three. Sky spread is 4.8 in the source and 4.8 with synthesis, 5.2 retained and 2.6 removed. Error: 19.1 retained, 35.0 removed, 55.5 synthesized. One measure of variation does not show that the grain's pattern, or its change over time, matches.
- Pixel error includes both grain patterns. The replacement grain is independent of the source's, so the error holds the missing original pattern and the new one, even when their statistics are alike. A high error does not by itself show that the grain looks wrong.
- At a coarse quantizer the cost gap nearly closes. At QP 34, retaining takes 255 levels to the clean picture's 237: most of the grain is quantized to zero either way. The sky is still uneven, 5.4 retained and 4.0 removed. With the grain removed, that spread comes from coding; retained, it may mix coding effects with surviving grain, which the spread cannot tell apart.
In the encoders
x264's grain tuning favors retention, and x264 has no grain synthesis for a decoder to run. AV1 has synthesis in the format: libaom's decoder adds the grain to a separate output image, after decoding.
The tuning's values, and libaom's parameters and options
- What the x264 tuning sets. The tuning sets deblocking −2:−2, smaller dead zones, no decimation of small coefficients, the psychovisual trellis term on, weaker adaptive quantization,
--qcomp0.8 and the I and B ratios at 1.1. What those do depends on the preset, the rate control mode and macroblock-tree, which sets the B ratio to 1 and takes over--qcomp. - What libaom's grain parameters hold. They include up to 14 points relating luma brightness to grain strength, filter coefficients that give grain its spatial shape, and a random seed. From libaom's option list only:
--denoise-noise-level, present when the build includes the denoiser, sets an amount of noise, with 0 for no denoising, and--film-grain-tablesupplies parameters from a file.
Trade-offs
- Retention keeps the original pattern as far as compression allows. It is the only one of the three that tries to reproduce the actual grain. How much survives depends on the quantizer: here the retained output already has an error of 19.1.
- Synthesis is cheap and approximate. It depends on a denoiser that removes grain without removing detail, and on a model that matches the grain's look. The demo's perfect denoiser hides the first risk.
- Pixel error penalizes replacement grain. Squared error measures fidelity sample by sample and cannot by itself judge whether synthesized grain looks convincing, a limit of quality metrics that the next unit takes up.
Sources
Read 2026-11-15, at the commits linked.
- x264 at commit c24e06c (GitHub mirror): the grain tuning in
common/base.c: deblocking, the trellis term, decimation off, the two ratios, adaptive quantization strength, dead zones, qcomp; inencoder/ratecontrol.c, the B ratio set to 1 with macroblock-tree; the--nroption, by its help line only. - libaom at commit 1b7f9ef (GitHub mirror): in
aom_dsp/grain_params.h, the luma scaling points, the filter lag and coefficients, and the seed; inav1/av1_dx_iface.c, grain added to a separate image and where the output image gets it; inav1/arg_defs.c,--denoise-noise-level, defined only in a build with the denoiser,--enable-dnl-denoisingand--film-grain-table, described from the option list. - Not checked: the AV1 specification's grain synthesis process; libaom's denoiser and how it estimates grain; whether the grain image is ever used as a reference (the code read builds it at output); grain tools in x265, SVT-AV1 and Versatile Video Coding; any viewing test.