Bitstream structure
An encoder's output is not one long string of coded blocks. It is a sequence of separately identifiable units, each with a small header that says what it holds: the settings a decoder needs first, a coded picture or part of one, or extra information. The unit layer helps a player find a place to start, skip what it cannot use and pick up after an error; decoding from a unit still depends on the settings and reference frames the decoder holds.
Wrap a payload as a unit
The payload is twelve invented bytes standing in for coded data. Choose a format and what the unit carries, and the demo wraps the payload the way that format's encoder does. H.264 and High Efficiency Video Coding (HEVC) call the unit a network abstraction layer (NAL) unit and, in a plain stream, mark where each begins with a start code: two or three zero bytes and a one. The payload must then never contain that pattern, so the encoder inserts an escape byte, 03, wherever two zeros are followed by a byte of 3 or less. AV1 calls the unit an open bitstream unit (OBU); in the form libaom writes, a size field takes the place of a start code. One choice of unit is a slice of an instantaneous decoding refresh (IDR) picture, a kind of keyframe.
For x264's sequence settings and the payload that contains 00 00 01 twice:
- The framed output is 19 bytes for a 12-byte payload: a four-byte start code, then a 15-byte NAL unit made of one header byte, the payload and two escape bytes.
- One byte says what the unit is. 0x67 is priority 3, type 7. This x264 IDR slice is 0x65, its P-frame slice 0x41 and its unreferenced B-frame slice 0x01: the same type as the P-frame, with priority 0, the reference indication (
nal_ref_idc) x264 gives a B frame nothing refers to. - Each 00 00 01 in the payload became 00 00 03 01. In the third payload the five zeros take two escape bytes and the 00 00 03 a third: an 03 already there after two zeros is escaped too, so a decoder can tell it from an inserted one. The payload with no zero runs takes none.
- HEVC differs in the header only: two bytes, 0x42 0x01 for these settings, and 20 bytes written.
- AV1 adds two bytes here and leaves the payload alone: 0x0A, then the length 0x0C, 14 bytes in all. The size field is one byte for a payload of up to 127 bytes and two from 128.
What the units are
The settings travel in parameter sets: a sequence parameter set (SPS), a picture parameter set (PPS) and, in HEVC, a video parameter set (VPS) above them. The numbers in the table are the type values in x264, x265 and libaom.
| Holds | H.264 | HEVC | AV1 |
|---|---|---|---|
| Settings for the sequence | SPS, type 7 | VPS 32, SPS 33 | Sequence header, 1 |
| Settings for pictures | PPS, type 8 | PPS, type 34 | In the frame header |
| A coded picture or part | Slice, 1 or 5 | Slice, 0 to 9, 16 to 21 | Frame, type 6 |
| Start of a time instant | Delimiter, 9 | Delimiter, 35 | Delimiter, 2 |
These are types of unit, not the I, P and B types of a slice's prediction.
The slice and frame types in detail
An H.264 NAL unit of type 5 carries a slice of an IDR picture: a keyframe at which x264 clears its list of references. HEVC defines more NAL unit types for coded slices; x265 lists values 0 to 9 and 16 to 21, and what each one means was not read for this page. For AV1 the table shows one way of packaging: libaom's list also has a frame header unit, 3, and a tile group unit, 4.
In the encoders
x264 repeats its parameter sets at every keyframe unless told not to, x265 only when asked, and libaom writes a sequence header at its key frames. A start code can give way to a length: x264 writes lengths when its output is a container.
Repeated settings, delimiters and container output, by encoder
- Where the settings are repeated. x264's default puts them at hand wherever a stream is joined at a keyframe; with open groups of pictures that keyframe can be an I frame with a recovery point, not an IDR picture. x265's
--repeat-headersis off by default. libaom's sequence header also goes before an intra-only frame. - Delimiters. They are optional in x264 and x265 (
--aud, off by default). libaom puts a temporal delimiter in front of the frame data it returns, for spatial layer 0 and when no earlier data is waiting to go out. - Container output. For MP4, Matroska or Flash video x264 turns start codes and repeated headers off and puts a four-byte length before each unit; the escape bytes stay.
Trade-offs
- Start codes let a reader find the next unit boundary from anywhere in a stream, at the cost of escape bytes. Decoding from there still depends on what the unit holds and on the settings and references at hand. Lengths cost nothing in the payload, but a reader that loses its place cannot resynchronize by itself.
- Repeating the settings costs bits at every keyframe and makes them available there without a container's help. Joining also needs a picture decoding can start from.
- In H.264 the header can mark a unit as droppable. A reference indication of 0 says no reference picture is rebuilt from the unit, so a reader can discard it without harming them; the picture it carried is lost. AV1's one-byte header says no such thing: the three frame choices above share one, and a frame's type and reference updates are written inside the payload.
Sources
Read 2026-11-15, at the commits linked. The lines on the non-IDR keyframe, x265's final byte, libaom's delimiter condition and its frame header were read 2026-11-16.
- x264 at commit c24e06c (GitHub mirror): in
common/bitstream.c,nal_escape_candx264_nal_encodewith the start code, the header byte and the length written in place of a start code; the unit types and priorities inx264.h; inencoder/encoder.c, which units get the four-byte start code, the IDR picture's type and priority, the I frame that is not IDR and its recovery point, the priorities of I and P frames and unreferenced B-frames, the delimiter and the parameter sets at a keyframe; the defaults incommon/base.c;--audand the container outputs inx264.c. - x265 at commit 4191822: in
source/encoder/nal.cpp, the start codes, the two header bytes and the escape rule and the 03 appended after a final zero byte; the unit types insource/x265.h; the defaults ofrepeat-headersandaud. - libaom at commit 1b7f9ef (GitHub mirror): the unit types in
aom/aom_codec.h; inav1/encoder/bitstream.c, the header byte, the size field and the sequence header at key frames and intra-only frames, and the frame header's frame type and reference updates; the length encoding; the temporal delimiter and the condition for writing it inav1/av1_cx_iface.c. - RFC 6184, section 1.3, read as extracted text: a reference indication of 0 means the unit is not used to reconstruct reference pictures and can be discarded without risking their integrity. The document describes carrying H.264 in packets; it is not the H.264 specification.
- Not checked: the specifications; what goes inside a parameter set, a slice header or a frame header; supplemental information units; the meaning of each HEVC unit type; the other framing arrangements AV1 allows, without a size field; Versatile Video Coding's units; how x265 and libaom write into containers.