How Video Compression Works: Lossy vs. Lossless and Artifact Reduction

Lossless vs. Lossy Video Compression

Lossless compression algorithms (like ZIP, PNG, or Apple ProRes RAW) mathematically condense data without discarding a single bit of information. When decompressed, the file is identical to the camera sensor's original output. However, lossless video results in enormous file sizes (hundreds of gigabytes per hour) that cannot be streamed over the web.

Lossy compression (H.264, VP9, AV1) strategically discards visual details that human eyes are least sensitive to, such as minute color gradations in shadows or rapid motion that occurs too fast for human retinal perception.

Spatial & Temporal Redundancy

Video compression achieves remarkable efficiency by targeting two types of redundancy:

1. Spatial Redundancy (Intra-frame): If a scene features a clear blue sky across the upper half of the screen, the encoder does not save millions of individual blue pixels. Instead, it defines a single color block and mathematical gradient vector.

2. Temporal Redundancy (Inter-frame): In a video of a person talking in front of a bookshelf, 95% of the pixels (the room and furniture) do not change from one frame to the next. The encoder only updates the moving pixels (the speaker's mouth and eyes).

Understanding Frame Types: I, P, and B Frames

Video encoders structure sequences into a Group of Pictures (GOP), consisting of three types of frames:

• I-Frames (Intra / Keyframes): Complete, standalone images (like a high-res JPEG). Every video must have periodic I-frames to allow users to seek or scrub through the timeline.

• P-Frames (Predicted): Store only what changed relative to the previous frame, using motion vectors.

• B-Frames (Bi-directional): Predict motion by looking both backward to previous frames and forward to upcoming frames, achieving the highest compression ratio.

Common Compression Artifacts Explained

When video is compressed too aggressively or encoded at insufficient bitrate, visual errors appear:

• Macroblocking: The image breaks into noticeable square pixel tiles during fast action scenes.

• Color Banding: Smooth gradients (sunsets, dark skies) separate into harsh stepped lines due to reduced color bit-depth.

• Mosquito Noise: Buzzing, fuzzy noise around sharp high-contrast text or edges.

• Ringing: Halo-like ripples surrounding thin lines caused by aggressive frequency filtering.

Rate Control: CRF, CBR and Two-Pass

Every encoder has to answer one question continuously: how many bits should this particular frame receive? The strategy it uses to answer is called rate control, and it affects the final result more than the choice of codec does.

Constant Rate Factor is the mode to reach for when you want a specific visual quality and do not care what the file size turns out to be. You give the encoder a number — for x264, values from 18 to 23 cover the range from visually transparent to clearly compressed — and it spends whatever bits each scene requires to maintain that standard. A static interview holds the quality cheaply; a confetti cannon costs a great deal. The file size is not knowable in advance, which is precisely the point: quality is pinned and size floats.

Constant bitrate inverts the trade, holding the data rate fixed and letting quality fluctuate. This is the right choice for live streaming, where the network has a known capacity you cannot exceed, and the wrong choice for a file you are keeping, because easy scenes get bits they do not need while hard scenes visibly fall apart. Two-pass encoding is the compromise for cases where you must hit a target size: the first pass analyses the whole video to find where the difficult material is, and the second distributes a fixed budget intelligently with that knowledge in hand. It takes twice as long and produces markedly better results than single-pass CBR at the same size.

Generational Loss: Why Re-Saving Degrades

Lossy compression is not idempotent. Running a file through an encoder a second time does not reproduce the same output; it discards a further slice of information, and the damage accumulates with every pass. This is generational loss, and it is the mechanism behind the familiar decay of a video that has been reposted across several platforms.

The reason it compounds is that the second encoder cannot tell the difference between detail that belongs to the original scene and artefacts introduced by the first encode. It faithfully spends bits describing the blocking and ringing it inherited, treating that damage as content worth preserving, while simultaneously discarding genuine detail to stay within budget. Each generation therefore gets both softer and more artefact-ridden at once, and no amount of raising the bitrate on a later pass recovers what earlier passes threw away.

The practical defence is to keep a master and to treat every lossy export as disposable. Edit from the highest-quality source you have, make all your changes in one editing session, and export once. When you need several variants — a high-resolution version, a social crop, a preview — generate each one from the master rather than from another export. And when a file only needs its container changed rather than its content re-compressed, remux instead of re-encoding: it is instant and it costs nothing.

• Always keep the highest-quality copy you have; export from it every time.
• Make all edits in one pass. Each export is a generation you cannot undo.
• Remux rather than re-encode whenever the codecs already fit the target container.
• Raising the bitrate on a later generation cannot restore discarded detail.

Measuring Quality: PSNR, SSIM and VMAF

Comparing two encodes by eye is unreliable and does not scale, so the industry uses objective metrics. They are worth knowing about because they disagree with each other in instructive ways.

Peak Signal-to-Noise Ratio is the oldest and simplest: it measures the mathematical difference between pixel values in the original and the encode. It is cheap to compute and only loosely connected to human perception, because it treats every pixel as equally important. An encoder that smooths away film grain scores well on PSNR while looking plastic. Structural Similarity Index improves on this by comparing local patterns of luminance and contrast rather than isolated pixels, which brings it closer to how people actually judge images.

VMAF, developed by Netflix, is the current practical standard. It combines several underlying metrics using a model trained on scores from real human viewers, and it was built specifically to predict perceived quality for streaming content. Its scale runs 0 to 100, where roughly 93 and above is generally indistinguishable from the source for normal viewing. The reason it matters outside a lab is that it lets platforms tune encoding ladders against perception rather than arithmetic, which is how streaming services deliver acceptable quality at bitrates that would have looked poor a decade ago.

Practical Tips for Preserving Visual Clarity

1. Always download the highest available source resolution. A 1080p stream downscaled to a 720p monitor looks sharper than a native 720p stream because downscaling increases perceived pixel density.

2. Avoid re-exporting video repeatedly. Each render step re-compresses the frames, compounding visual artifacts.

3. Choose modern codecs (VP9 or AV1) whenever bandwidth is limited.