Inanely Fast Local AI: 775 token per second! 🤯 I was able to run the new DiffusionGemma (full BF16 model) by @googlegemma on vLLM (fork by Red Hat) on Nvidia RTX 6000 Pro. It's blazing fast at short contexts, but gets slow very quickly. At 100k, TTFT is 22s! ■ Leave a comment https://t.co/48SWaVTDK4
Behnam
·
2026-06-12
·
via Hacker News - Newest: "AI"
Article URL: https://twitter.com/OrganicGPT/status/2064883777499795716 Comments URL: https://news.ycombinator…
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。