CUDA-Oxide 0.1, RTX 5070 Launch, & BeeLlama.cpp Boost 3090 Inference

Today's Highlights

NVIDIA makes strides in developer tools with a Rust-to-CUDA compiler, while ZOTAC quietly launches an RTX 50 series GPU. Meanwhile, a new llama.cpp fork pushes local LLM inference speeds and VRAM efficiency on consumer hardware.

NVIDIA releases CUDA-Oxide 0.1 for experimental Rust-to-CUDA compiler (r/nvidia)

Source: https://reddit.com/r/nvidia/comments/1t7a7e7/nvidia_releases_cudaoxide_01_for_experimental/

NVIDIA has officially released CUDA-Oxide 0.1, an experimental compiler designed to translate Rust code into NVIDIA's PTX (Parallel Thread Execution) assembly. This project aims to bring the memory safety guarantees, modern language features, and robust tooling ecosystem of Rust to high-performance GPU computing, offering a compelling alternative to traditional CUDA C++ for systems-level programming on GPUs. CUDA-Oxide targets the existing CUDA ecosystem, enabling Rust developers to leverage NVIDIA's powerful GPUs for highly parallel processing tasks without sacrificing performance-critical optimizations or requiring a complete paradigm shift. The initial release marks a significant step towards broadening the accessibility of GPU programming, enabling a wider range of software engineers to contribute to CUDA-accelerated applications, and potentially improving code reliability and maintainability in complex HPC and AI workloads by reducing common pitfalls associated with C++ memory management. This initiative could foster a new generation of CUDA kernels written in Rust, benefiting from its strong type system and ownership model.

Comment: As a developer, I'm excited about using Rust for CUDA. The promise of memory safety and modern language features in GPU kernels could drastically reduce bugs and improve productivity for complex parallel tasks, especially for new projects.

ZOTAC quietly launches GeForce RTX 5070 AMP GPU, its first RTX 50 AMP model in white (r/nvidia)

Source: https://reddit.com/r/nvidia/comments/1t8ddk8/zotac_quietly_launches_geforce_rtx_5070_amp_gpu/

ZOTAC has quietly introduced its first GeForce RTX 50 series graphics card, the RTX 5070 AMP, distinguished by its unique white aesthetic and ZOTAC's signature factory-overclocked performance. This subtle launch, observed from a major NVIDIA partner, signals the gradual rollout of NVIDIA's next-generation Blackwell GPU architecture into the consumer market. While specific performance benchmarks and architectural details for the RTX 5070 are not yet officially disclosed, its emergence suggests significant improvements in core performance, advanced ray tracing capabilities, and enhanced power efficiency over its 40-series predecessors. The 'AMP' designation typically implies a premium offering, featuring robust power delivery and advanced cooling solutions designed to extract optimal out-of-the-box performance and stability for enthusiasts. This release sets the stage for more widespread RTX 50 series announcements and detailed performance analyses in the coming months, offering an exciting glimpse into the future of high-end gaming and professional GPU capabilities built on NVIDIA's latest silicon roadmap. It confirms that the next generation of GPUs is indeed on the horizon for consumers.

Comment: A quiet launch of an RTX 5070 is intriguing. It hints at the impending full Blackwell rollout and suggests NVIDIA partners are readying their custom designs, even if official specs are still under wraps.

BeeLlama.cpp: advanced DFlash & TurboQuant with support of reasoning and vision. Qwen 3.6 27B Q5 with 200k context on 3090, 2-3x faster than baseline (peak 135 tps!) (r/LocalLLaMA)

Source: https://reddit.com/r/LocalLLaMA/comments/1t88zvv/beellamacpp_advanced_dflash_turboquant_with/

BeeLlama.cpp, a newly unveiled fork of the widely adopted llama.cpp inference engine, is making waves with its introduction of advanced optimization techniques, specifically DFlash and TurboQuant. These innovations are engineered to significantly boost performance for local Large Language Model (LLM) inference on consumer-grade GPUs. This fork has already demonstrated impressive capabilities, successfully running the Qwen 3.6 27B Q5 model with an unprecedented 200,000 token context on a single NVIDIA RTX 3090 GPU, all while achieving peak generation speeds of 135 tokens per second. Such optimizations contribute to a remarkable 2-3x speedup compared to the baseline llama.cpp implementation, making high-context and larger models far more accessible and practical on existing hardware. Beyond raw speed, BeeLlama.cpp also integrates robust support for reasoning and vision capabilities, actively pushing the boundaries of what is achievable for multimodal local inference. This project provides an invaluable, hands-on tool for developers and AI enthusiasts aiming to maximize their GPU's potential for demanding AI applications, particularly those focused on long-context processing, complex reasoning, and multimodal inputs directly on their local machines.

Comment: Seeing a llama.cpp fork achieve 200k context at 135 tps on a 3090 is a game-changer for local LLM users. DFlash and TurboQuant seem like crucial VRAM and speed optimizations.