Google Cloud inks AI infrastructure deal with Thinking Machines

Thinking Machines Lab Inc. will expand its use of Google LLC’s cloud platform as part of a partnership the companies announced this morning.

TechCrunch cited a source as saying that the deal is valued in the “single-digit billions”.

Thinking Machines is an artificial intelligence startup led by Chief Executive Mira Murati, the former Chief Technology Officer of OpenAI Group PBC. It offers a cloud service called Tinker that enables developers to create custom versions of open-source large language models. Tinker performs customization by attaching add-ons to LLMs’ core code.

Thinking Machines will move some of its workloads to Google Cloud’s A4X Max instances, which are specifically optimized for AI models. Each virtual machine provides access to 4 of Nvidia Corp.’s Blackwell Ultra graphics processing units. They’re supported by 2 central processing units that feature 72 cores apiece.

The chips run in liquid-cooled GB300 NVL72 appliances. The systems, which are made by Nvidia, come with 37 terabytes of memory and 130 terabits per second of internal bandwidth for moving data between processors. According to Google Cloud, Thinking Machines is one of the first customers to use its NVL72 infrastructure.

The chips in A4X Max instances use a technology called RoCE to exchange data. It skips several of the steps usually involved in processing packets, which boosts throughput. The data travels over a network that features a so-called rail-aligned topology. 

Packets often have to travel through multiple network devices before reaching their destination. Each such device is known as a hop. A rail-aligned topology creates dedicated network links, or rails, between GPUs that minimize the number of hops data must go through, which boosts workload performance.

Google uses Nvidia’s ConnectX network interface cards, or NICs, to coordinate GPU traffic. A NIC is a chip that acts as an interface between a server and the data center network to which it’s attached. The ConnectX chips run alongside Google’s internally-developed Titanium NICs. A4X Max instances use the latter modules to process traffic between GPUs and external systems such as other Google Cloud services.

Thinking Machines is already using several of those services to run its workloads. The company keeps information in Google Cloud Storage, the Spanner relational database and a custom cache. It relies on a fourth Google Cloud service called Cluster Director to automatically fix certain technical issues.

The partnership comes a few weeks after Thinking Machines disclosed a “significant investment” from Nvidia. The cash infusion is part of a broader deal that will see the startup purchase billions of dollars worth of hardware. One of the Nvidia products that Thinking Machines plans to adopt is Rubin, the successor to the Blackwell Ultra graphics card that powers Google’s A4X Max instances.

Image: Google