An AMD engineer has contributed to the upstream FFmpeg library an ONNX Runtime back-end for its DNN filter. The FFmpeg Deep Neural Network (DNN) filters allow for running AI models natively inside the video processing pipeline for upscaling, object detection, background segmentation, and more. This ONNX Runntime back-end support is notable in that it expands the GPU and NPU capabilities with FFmpeg.

AMD engineer Steven Xiao authored the ONNX Runtime back-end for FFmpeg's DNN processing filter. This addition opens up FFmpeg DNN to supporting inferencing on multiple GPU and NPU platforms. Beyond the CPU execution, there is NVIDIA CUDA support, Windows DirectML execution across all the major GPU vendors, and also AMD Ryzen AI NPU support across platforms using the ONNX Runtime VitisAI execution provider. The main takeaway is AMD beginning to make the Ryzen AI NPU useful for FFmpeg.

Those interested can learn more about this ONNX runtime back-end for the FFmpeg DNN filter via this commit.