British inference chip startup Fractile bags $220M to accelerate token consumption

U.K.-based artificial intelligence inference chip startup Fractile Ltd. said today it has closed on a $220 million Series B round of funding. The round was co-led by Accel, Factorial Funds and Founders Fund, and saw the participation of Conviction, Gigascale, O1A, Felicis, Buckley Ventures, 8VC and existing backers.

The company was founded in 2022 by the Oxford University-trained chip engineer Walter Goodwin (pictured), who serves as its Chief Executive. He helped to design Fractile’s specialized inference chips, targeting workloads powered by trained AI models.

According to Goodwin, the company is targeting what it believes is the key constraint facing the world’s most advanced frontier models, namely the time it takes for them to generate outputs after being prompted. As AI models grow in size and become increasingly sophisticated, they now require tens of millions of “tokens,” which are the basic unit of measurement in advanced computing, to solve hard problems or complete assigned tasks. However, these tokens require lots of data to be moved between the processor and its associated memory, which increases the latency, and therefore the response time to queries.

To get around this, Fractile has developed a novel logic chip complete with an architecture that attaches memory to fit inside a standard server rack. The design helps to reduce latency and maximize bandwidth without sacrificing speed, the company said. While Fractile has kept the technical specifications of how its chips work close to its chest, Goodwin told the Wall Street Journal that it doesn’t use traditional high-bandwidth memory, nor on-chip static random-access memory or SRAM. That suggests it’s based on an entirely novel design.

According to Goodwin, Fractile’s chips can help to dramatically accelerate AI workloads, and also enable entirely new ones that aren’t possible on standard graphics processing units, which are the most widely used AI chips today.

“Compressing a month of work into a day, a weekend of lab computation into a coffee break, will make all that work happen radically faster, but it will also make far more ambitious AI use cases economically viable,” Goodwin wrote in a blog post announcing the round. “The defining work of the 21st century will be marked by the engine of inference delivering immense and diffuse chains of intellectual inquiry, in drug discovery, in software engineering, in materials discovery, in any field where humanity will benefit from sheer intellectual work to resolve complex problems.”

Such tantalizing claims will be put to the test in due course, and Fractile will need to show that it can live up to them if it wants to stand out in an increasingly competitive market for specialized inference chips. In recent years, a host of inference chip specialists have emerged, hoping to take market share away from Nvidia Corp., the AI chip market leader.

Those rivals include Cerebras Systems Inc. and its dinner plate-sized WSE-3 chips, which also target AI production workloads. Cerebras is set to go public tomorrow via an initial public offering that will raise at least $5.5 billion, the industry’s biggest public debut in years.

Other rivals include SambaNova Systems Inc., which recently cemented a partnership with Intel Corp., and startups such as Untether AI Inc. and Graphcore Inc. It’s also going up against Nvidia itself, which recently launched the Groq 3 language processing unit for inference workloads, and public cloud giants like Amazon Web Services Inc. and Google Cloud, which also have dedicated inference chips. OpenAI Group PBC could be another rival, as it’s currently believed to be working with Broadcom Inc. and Taiwan Semiconductor Manufacturing Co. Ltd. on its own inference chip design.

Photo: Fractile