INFRA

AI networking provider Upscale AI raises $190M at $2B valuation

Data center networking startup Upscale AI Inc. today announced that it has raised $190 million in fresh funding.

The capital was provided as an extension to a $200 million Series A round that the company first closed in January. The new raise was led by Premji Invest with participation from Nvidia Corp., Salesforce Ventures, Seligman Ventures, Temasek and several existing backers. Upscale AI is now valued at $2 billion.

A data center includes several types of switches that are each designed for different tasks. There are scale-up switches optimized to move data between servers installed in the same rack. Traffic between racks, in turn, is managed by so-called scale-out devices. Upscale AI is developing scale-up and scale-out switches optimized for artificial intelligence workloads.

The calculations that an AI model uses to process a prompt must be carried out one after one another. If an unexpected latency spike delays one of the calculations, all the subsequent computations have to be postponed. Such inefficiencies can emerge when graphics processing units exchange data more slowly than expected while running an AI model. 

Upscale AI says that its scale-up switches series addresses the challenge by providing deterministic latency. Data movement speeds can be foreseen in advance, which avoids unexpected delays that can mix up calculations. The feature is powered by a custom chip the company calls SkyHammer.

SkyHammer supports multiple open-source network protocols optimized for scale-up traffic. One of the supported technologies, UALink, enables GPUs to access data in one another’s memory as if it were local RAM. SkyHammer is also compatible with ESUN, a version of the popular Ethernet protocol optimized for AI workloads. 

In March, Upscale AI previewed a line of scale-out switches for linking together graphics card racks. The product line is based on Nvidia’s Spectrum-X chip series. The processors power an eponymous lineup of Ethernet switches that the chip giant sells alongside its GPUs.

Traffic traveling between two graphics cards must usually go through a central processing unit before reaching its destination. Nvidia’s Spectrum-X switches support a technology called RoCE that enables packets to bypass the CPU, which reduces latency. The devices also collect telemetry to help administrators detect technical issues.

Upscale AI’s scale-out switch series will combine Spectrum-X silicon with an open-source operating system called SONiC. The software, which was originally developed by Microsoft Corp. to power its public cloud, streamlines many network management tasks. However, using it can still be challenging. Upscale AI has developed an AI-optimized version of SoNIC designed to simplify the user experience.

“AI infrastructure is being redefined at cluster scale, and networking is one of the most critical bottlenecks,” said Upscale AI Chief Executive Officer Barun Kar. “Upscale AI is building a high-performance, open-standard AI fabric purpose-built for large-scale, synchronized workloads.”

The company says that its hardware is currently being evaluated by multiple hyperscalers and neocloud operators. Upscale AI will use its newly raised capital to accelerate its commercialization efforts.

Image: Unsplash