惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

博客园 - 叶小钗
Last Week in AI
Last Week in AI
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
V
V2EX
J
Java Code Geeks
The GitHub Blog
The GitHub Blog
博客园_首页
U
Unit 42
人人都是产品经理
人人都是产品经理
Engineering at Meta
Engineering at Meta
IT之家
IT之家
G
Google Developers Blog
L
LangChain Blog
钛媒体:引领未来商业与生活新知
钛媒体:引领未来商业与生活新知
爱范儿
爱范儿
罗磊的独立博客
博客园 - 三生石上(FineUI控件)
Martin Fowler
Martin Fowler
Jina AI
Jina AI
有赞技术团队
有赞技术团队
Apple Machine Learning Research
Apple Machine Learning Research
Vercel News
Vercel News
小众软件
小众软件
H
Help Net Security

Hacker News - Newest: "AI"

AI can't read an investor deck AI as an attorney? Student uses ChatGPT, Gemini to sue UW over alleged racial discrimination Hacking MCP Servers in AI Systems – The Rug Pull: Tool Changes After Approval GitHub - MeepCastana/KubeezCut: Free Web based video editor Can AI judge journalism? A Thiel-backed startup says yes, even if it risks chilling whistleblowers Coming soon: 10 Things That Matter in AI Right Now DARPA built an AI to fact-check enemy weapons claims What explains heterogeneity in AI adoption? When AI Meets Muscle: Context-Aware Electrical Stimulation Promises a New Way to Guide Human Movements - Department of Computer Science AI Changed How We Build. It Did Not Change What Matters. Linux rules on using AI-generated code - Copilot is OK, but humans must take 'full responsibility for the… Meta spins up AI version of Mark Zuckerberg to engage with employees Code Mode: Let Your AI Write Programs, Not Just Call Tools | TanStack Blog GitHub - Delavalom/graft: Go framework for building AI agents. Type-safe tools, multi-provider (OpenAI, Anthropic, Gemini, Bedrock), zero vendor SDKs. India's TCS tops estimates, says new AI models did not dent services demand Gen Z's fading AI hype Strong feeling: we are in a folded AI reality GitHub - machinarii/total-recall-catalog: A reference catalog of latest knowledge retrieval, memory & RAG systems GitHub - mensfeld/code-on-incus: Give each AI agent its own isolated machine with root, Docker, and systemd. Active defense detects and stops threats automatically.. Quantization, LoRA, and the 8% Problem: Benchmarking Local LLMs for Production AI Iran war: We spoke to the man making Lego-style AI videos that experts say are powerful propaganda Powell, Bessent discussed Anthropic's Mythos AI cyber threat with major U.S. banks GitHub - immartian/bellamem: Persistent belief-graph memory for AI agents. Retrieves decisive context by importance — not recency, not RAG, not /compact. recursive-mode: The Repo-Native Operating System for AI Engineering After the attack on Sam Altman's home, will AI CEO's go on the offensive? The biggest advance in AI since the LLM Opus 4.6 vs GPT 5.4 One Prompt Unity World Generation Test “AI polls” are fake polls Client Challenge Can AI be a 'child of God'? Inside Anthropic's meeting with Christian leaders
Lexar Wants to Offload Local AI Models to SSD Amid the RA...
by AleksandarK · 2026-06-16 · via Hacker News - Newest: "AI"

Lexar has been experimenting with various technologies to help consumers achieve faster data throughput and more reliable storage. However, the company is now envisioning something entirely different as the PC evolves from a regular personal computer to a local AI-enhanced experience. We had the opportunity to interview Lexar's Chief Technical Officer (CTO), Daniel Guo, about the technology Lexar is developing to help offload some of the DRAM demand to much cheaper NAND Flash. According to Guo, DRAM is about six times more expensive to manufacture than NAND Flash, and there are opportunities for AI SSDs to reduce the DRAM requirements for running AI models on local hardware. This is where the Lexar AI Storage Core SSD comes into play, as the company is creating new storage solutions for consumers to support local AI deployments using much less DRAM by offloading large language models (LLMs) to SSDs. This approach allows larger and more powerful LLMs to fit into a PC build, reducing memory footprint by at least 40%

Based on internal testing, Lexar managed to run the Qwen 3.5 122B AI model on a local PC. Traditionally, users would need to spend about $4,500 on a PC with a decent CPU and 128 GB of DRAM to run this model. Through hardware and software optimization, the Lexar AI suite with the Lexar AI Storage Core SSD can reduce the DRAM requirement to 32 GB and run the model with 35 billion parameters at 15.6 tokens per second, compared to only 5.2 tokens per second using traditional frameworks. When attempting to load the 122B model on 32 GB of DRAM, the traditional Llama.cpp fails to load and crashes, while Lexar's SSD offloading provides about 4.4 tokens per second.

When the system is equipped with a more robust configuration featuring 64 GB of DRAM, running the 122B model with a larger context window is only possible with SSD offloading. With about 4,000 tokens in context, both traditional configurations and the Lexar AI stack run at a slightly higher speed. However, for larger contexts, often needed at 256K tokens, only the Lexar AI suite can launch and manage to produce about 19.3 tokens per second. Of course, this doesn't mean the setup is perfect, and not every model size can be offloaded to the SSD. With larger LLMs, system latency increases significantly, as the time between submitting a prompt and receiving a response grows exponentially.

The time to first token, often called TTFM, has been measured at about two seconds before the first token appears after the prompt is submitted with a 2K context window. When the context is larger at 4K, the delay increases to anywhere between 6 and 8 seconds. Technically, users could offload models that are about 400 billion parameters large, but the tokens per second and TTFM would be very slow. For some, this might be suitable, but for others, buying more DRAM is the better solution. Either way, this is an intriguing concept from Lexar.

Example from Computex 2026.
The company developed a concept for Mini-PCs and desktops featuring an M.2 slot designed for multiple insertions. An M.2 SSD is encased in a metal jacket (not a full enclosure) and is inserted into a 25 mm-wide slot on the front panel of a mini PC, connecting directly to the M.2 slot wired to the processor or chipset. This design eliminates other overheads. The hot-swappable SSD, which offloads AI models onto NAND Flash, reduces dependency on DRAM and aids in running larger models. It is available in both PCIe Gen 5 and Gen 4 versions, with the Gen 5 version offering more bandwidth. This M.2 SSD uses Lexar's custom Storage Processing Unit (SPU) DRAM-less controller for complete control over data movement.