Atop a lab bench, Cornell Tech postdoctoral researcher Yifan He positions the lens of an optical receiver almost a meter away from an LED emitting a beam of red light. The computer monitor attached to the receiver takes a beat to refresh, then displays an array of squares that resemble a QR code.

When you hold your phone camera up to a QR code, light strikes the image sensor as only a first step to revealing the data hidden behind the black and white matrix. The receiver here is doing something different: Directly altering its own memory using the photocurrents produced by the beamed array of light. And unlike the data behind a QR code, which might point to a simple web address, this optical code could convey the parameters of an AI model.

The new receiver design, presented last month at the IEEE/JSAP Symposium on VLSI Technology & Circuits, seeks to reduce the burden of increasing memory demands on AI systems. Shining data down onto processors could lower the energy typically required for data centers, self-driving cars, and even “edge” applications like AI-powered robots, researchers say.

“People are designing all sorts of different AI chips,” says Jae-sun Seo, an associate professor of electrical and computer engineering at Cornell Tech, in New York City. These processors don’t often have room for all the parameters that make up AI models, so the additional data is stored in dynamic random-access memory (DRAM). The electrical connections commonly used to move the data between the DRAM and the processor create cost and efficiency concerns when systems scale up. “That’s one of the major bottlenecks.”

Optical links move data at high bandwidth with less energy loss than metal wires, but today’s optical receivers undercut that advantage by relying on power-hungry analog circuits to convert light to electronic bits. The group’s new tech would instead receive rapid flashes of digital QR code-like matrices so that chips can tweak model parameters without those analog circuits, enabling fully digital optical communication that would consume less energy.

“This is a really important problem,” says Dennis Sylvester, an IEEE Fellow who chairs the University of Michigan’s electrical and computer engineering department and was not involved in the work. “It’s got massive commercial implications. This solution is a clever way of dealing with it.”

Jae-sun Seo [left] and Yifan He [right] have developed a receiver that can edit memory in response to QR-code-like arrays of light.Alex Music

How light “flips” memory to power AI

Processors have a bit of built-in static random-access memory (SRAM), but not enough to allow an AI model to run independently. While SRAM is the faster of the two memory options, DRAM can store more data in the same footprint.

In the new system, the DRAM sits with the transmitter and the receiver is part of the processor’s SRAM. The transmitter beams the data to the array of SRAM cells, which in this case are modified to contain photodiodes. Light hitting each photodiode creates a current to flip binary values in the SRAM.

Creating a link between the light and receiver requires calibration, because you can’t expect them to be perfectly aligned or perpendicular to each other. So the chip references a data frame that has information about the expected position of each pixel of data, and uses that reference to ensure it can receive the real data, He says. “Ideally the best way is to have direct, point-to-point space between the transmitter and the receiver,” Seo adds, “but even if it’s slightly tilted, we have this calibration circuit.”

For applications in real-world settings, the researchers say they will need to build an optical transmitter that can alter the light matrix millions of times per second, transferring gigabits per second. The transmitter I saw in He and Seo’s lab is only a proof of concept, emitting a static 14x14-bit matrix through a metal mask over the light. The researchers say they are working with optics research groups to build a transmitter that is capable of rapidly changing the matrix.

The future of light-based memory links

Sylvester says that the tech in its current form is likely far from commercialization, owing to the fact that the individual photosensitive bit cells are larger than SRAM bit cells in conventional chips. Those larger cells mean the chip can fit less memory, a trade-off that he says could cancel out the added efficiency of the light-based approach.

Seo says that it’s part of the group’s ongoing efforts to shrink the bit cells, which can be achieved by optimizing the size of transistors and circuits and leveraging CMOS scaling.

Seo and He are looking at uses for the tech in robotics and other edge applications. One example is in AI robot-powered warehouses and factories, which could use optical data transmission to save time and energy when updating the AI models in each robot. Additionally, microrobots, which are inherently memory-constrained due to their size, could one day benefit from the tech, though it would require a more size-conscious design.

“Edge AI is a big growth area, and in three, four, five years, you’re going to hear as much about that as you are with data centers, probably, as the intelligence migrates more and more into these devices that we have,” Sylvester says.