A critical vulnerability has been disclosed in FFmpeg’s MagicYUV decoder that allows attackers to weaponize seemingly harmless media files and, in some scenarios, achieve remote code execution (RCE).

The flaw, tracked as CVE-2026-8461 and dubbed “PixelSmash,” is a heap out-of-bounds write in FFmpeg’s libavcodec component, with a CVSS score of 8.8 (High).

According to the JFrog Security Research, a single crafted AVI, MKV, or MOV file is enough to crash applications or, with a refined exploit chain, execute arbitrary commands on the underlying system.

FFmpeg is one of the most widely deployed media processing frameworks and is bundled into countless applications, including desktop video players, Linux thumbnail generators, self-hosted media servers, cloud transcoding pipelines, and even AI/ML data processing stacks.

Because the MagicYUV decoder is enabled by default in upstream FFmpeg builds and most major Linux distributions, the bug propagates silently into any project that links libavcodec.

FFmpeg Vulnerability

JFrog confirmed crashes against applications such as Kodi, mpv, ffmpegthumbnailer, Jellyfin, Emby, Nextcloud, Immich, PhotoPrism, and OBS Studio, and demonstrated full RCE against a Jellyfin media server and a Nextcloud instance using a malicious 50 KB AVI file.

The root cause lies in how the MagicYUV decoder handles video slices and chroma plane heights. MagicYUV uses horizontally divided slices per frame, and for subsampled formats like YUV420P, the decoder must translate luma slice heights into chroma slice heights.

Due to a rounding mismatch between the frame allocator and the decoder, an attacker-controlled slice_height value can cause FFmpeg to write one full row of chroma data beyond the end of a heap-allocated buffer.

In the proof-of-concept, a crafted media stream sets slice_height to an odd value, leading to an accumulation of off-by-one rows that push writes directly into adjacent heap structures.

Critically, the overflowing writes land on an AVBuffer structure that FFmpeg uses for reference-counted frame buffers.

By crafting a malicious payload, an attacker can overwrite FFmpeg memory structures, causing a call to system() with an attacker-controlled command and enabling remote code execution.

In JFrog’s Jellyfin exploit, this was weaponized into a reverse shell that executed as the Jellyfin service account once the media library scan triggered ffprobe on the malicious file.

The attack surface is broad and often “near-zero-click.” On desktops, simply browsing to a folder can trigger thumbnail generation via ffmpegthumbnailer and activate the bug.

On servers, media platforms like Jellyfin, Emby, and Nextcloud automatically invoke ffmpeg or ffprobe to generate previews and metadata when new files appear or are viewed.

JFrog also warns that similar vectors exist in cloud media pipelines and AI/ML environments that use FFmpeg (directly or via PyAV/OpenCV) to decode user-supplied video.

Turning PixelSmash into a potential denial-of-service or exploitation primitive inside shared inference or data-processing workers.

To exploit PixelSmash, an attacker only needs to deliver the crafted media file to any application that decodes video using FFmpeg with MagicYUV enabled.

No authentication or elevated privileges are required beyond the ability to upload, share, or place a file for automatic processing.

This makes common setups, such as torrent clients writing directly to media library directories, especially dangerous, as Jellyfin-style auto-scans can process malicious content immediately after download, without any user interaction.

The vulnerability has been fixed in a patched FFmpeg release (e.g., 8.1.2 or later with the MagicYUV bounds checks applied), and users are strongly urged to upgrade their FFmpeg builds as soon as possible.

As temporary mitigation, administrators can rebuild FFmpeg with the MagicYUV decoder disabled or apply the minimal patch that rejects invalid slice_height values before decoding.

Because this is a supply chain issue in a foundational library, projects that embed FFmpeg are advised to audit their builds, reduce enabled codecs where possible, and adopt allow-listing strategies similar to Plex’s minimal decoder configuration to limit the blast radius in the future.

Follow us on Google News, LinkedIn, and X to Get More Instant Updates.