AI and ML

Jalapeño is the latest announcement that attempts to portray OpenAI as more than a race-to-the-bottom model maker

OpenAI and Broadcom have teamed up – with a little help from some of the former’s AI models – to develop the frontier model lab’s very first inference chip, dubbed Jalapeño, the companies announced in a press release on Wednesday. 

Details of the spicily named silicon are scarce in the announcement, with the company admitting that it’s running engineering samples of Jalapeño in its lab “at target frequency and power,” but noting that it won’t have any technical details to share until a report on its performance is released in the coming months. But that doesn't stop it from claiming that "early testing shows that Jalapeño will deliver performance per watt substantially better than current state-of-the-art." Alright then! 

OpenAI said that it designed Jalapeño itself, with Broadcom serving as its implementation and integration partner (i.e., they made the darn thing), but it wasn’t humans alone who helped come up with Jalapeño’s made-for-inference ASIC architecture: AI helped, natch, and the result is what OpenAI says is the fastest ASIC development cycle ever in the high-performance advanced semiconductor space.

“Jalapeño was co-developed from initial design to manufacturing tape-out in just nine months,” OpenAI claimed. “That speed reflects deep software-hardware co-development with OpenAI’s engineering teams, Broadcom’s silicon implementation expertise, and the use of OpenAI models to accelerate parts of the design and optimization process.”

In other words, AI is now helping design the chips it’ll run on. Here’s hoping they ironed out the hallucinations before heading to production. 

OpenAI explained in the announcement that Jalapeño is just the first of its AI accelerators, with the chip serving to define its “vision for the future of LLM inference,” and one that will involve OpenAI controlling the entire stack behind its models and products. 

According to the release, OpenAI envisions a future where it doesn’t just own the frontier models and the products built on top of them, but the infrastructure underneath as well.

“Chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and product experience” are all part of OpenAI’s full-stack vision, which it said will enable it to make models “faster, more reliable, and more affordable.” 

And more locked in, one would assume, like a proverbial walled garden. The Apple of the AI world, if you will.

OpenAI is far from alone in developing its own silicon to help power AI – most of the giants in the space, including Amazon, Google, Meta, and Microsoft, have been building and using their own silicon for AI for several generations now, and OpenAI arch-rival Anthropic is reportedly considering a similar move. 

No telling, either, how OpenAI is intending to continue funding this capital-intensive initiative, given that it ran an operating loss of over $20 billion last year, according to leaked financials reported by Ed Zitron, and has apparently committed massive amounts ($600 billion? $1.4 trillion?) to infrastructure spending over the next few years.

But hey – if we questioned AI economics, nothing would ever get built, would it? ®