Oxford-based Lumai has launched the world’s first optical computing system that can run a billion-parameter large language model (LLM) in real time. Dubbed Iris Nova, this server can help companies deliver artificial intelligence (AI) inference services while consuming up to 90 percent less power.

Standard systems use silicon for computation, which, for some time, has been nearing the limits for its efficiency, which has a major impact on its scalability and power consumption. Each new generation of traditional processors now is able to deliver more compute, but only at an exponential increase in energy consumption. 

Data centers being built to serve the upcoming AI era are expected to consume more power than ever before. Estimates suggest that global power demand will double by the end of this decade as companies race to meet growing demands for AI inference. But Oxford-based Lumai believes it does not have to be so, thanks to their optical computing system, which is scalable and ready for deployment right away. 

Optical computing system

With diminishing returns on silicon-based computing, Lumai has turned its attention from electrons to photons. The company suggests that this simple shift allows it to deliver an order of magnitude increase in computing performance while delivering significant energy savings. 

Lumai’s Iris Nova is a server that uses light to bypass the constraints of two-dimensional silicon chips, achieving massive spatial parallelism. The server deploys a hybrid processor that uses an optical tensor engine for mathematical computation and digital processing for system control. 

The company, a spin-off from the University of Oxford, has been working on optical computing with a clear mission to meet escalating demand for computing with AI’s surge. By using light in a three-dimensional volume, the system can perform millions of operations simultaneously. 

Lumai’s optical processors enable faster inference and higher execution efficiency, delivering significant energy savings. According to the company’s estimates, Lumai servers deliver the same performance as conventional systems while consuming up to 90 percent less power.  

Billion-parameter LLM in real time

To demonstrate its capabilities, Lumai ran inference on Llama 8B and 70B in real time recently using its hybrid digital processor on the Iris Nova server. This also demonstrates that the servers can integrate intodata centers seamlessly and be adopted into existing workflows. 

Lumai has made the Nova server available for evaluation to hyperscalers, neo-clouds, enterprises, and research institutes. However, the company won’t stop here. Its lineup also includes two more servers, Aura and Tetra, which undoubtedly offer more capacity and can deliver AI inference more sustainably than GPU-based systems. 

“The demands on existing AI processors necessitate an urgent search for alternative scaling pathways,” said Suraj Bramhavar, Program Director at ARIA, a UK-government-backed agency. “Lumai is leading the charge in demonstrating that optical processors could provide one such pathway, and Advanced Research and Invention Agency (ARIA) is excited to partner with them to explore the shift beyond our traditional digital computing paradigm.”

“As the industry transitions into the inference era, we are simultaneously crossing the threshold into the post-silicon era,” added Xianxin Guo, CEO and Co-Founder of Lumai, in a press release.