Standard Intelligence raises $75M to develop efficient computer use models

Standard Intelligence Inc., a six-person artificial intelligence startup, today announced that it has raised $75 million in funding.

Sequoia and Spark Capital led the round. They were joined by multiple angel visitors including prominent AI researcher Andrej Karpathy.

Standard Intelligence has developed a foundation model called FDM-1 that is specifically optimized for computer use tasks. Those are tasks that require an AI to interact with an application via its graphic interface. According to Standard Intelligence, FDM-1 can perform a wide range of activities ranging from scanning software for vulnerabilities to using computer-aided design programs.

Computer use models are typically trained on screenshots of humans interacting with applications. Those images have to be manually annotated with explanatory notes. For example, a series of screenshots that depicts an online shopping workflow must be accompanied by a natural language description of the purchase. 

Standard Intelligence trained FDM-1 on video footage instead of screenshots. Additionally, it replaced human-crafted annotations with a so-called inverse dynamics model, or IMD. That’s a neural network capable of generating screenshot explanations automatically. 

Generating annotations with an IMD costs significantly less than doing so manually. As a result, researchers can create larger training datasets than would otherwise be possible. Standard Intelligence has assembled a computer use training dataset with 11 million hours of footage, which is multiple orders of magnitude bigger than the next best open-source alternative.

Increasing the size of an AI model’s training dataset boosts its output quality. One demo video posted by Standard Intelligence shows FDM-1 designing a metal component using a popular engineering application. In another test, the company’s engineers made the controls of an autonomous vehicle accessible through a website. FDM-1 learned how to drive the vehicle via the website after one hour of fine-tuning.

Another selling point of the model is that it uses relatively limited hardware resources. One contributor to its efficiency is that it doesn’t rely on chain-of-thought reasoning or tools to perform tasks in applications. Additionally, FDM-1 features a video encoder that Standard Intelligence says is 100 times more efficient than OpenAI Group PBC’s alternative.

A video encoder is a software module tasked with translating footage into mathematical representations that AI models can understand. Those mathematical representations can require a significant amount of memory to store. It’s possible to reduce their hardware footprint, but doing so usually decreases AI output quality.

FDM-1’s encoder mitigates that trade-off using a so-called masked compression objective. The technology removes unimportant parts of the footage that the model processes, which reduces memory requirements without lowering data quality. According to Standard Intelligence, the encoder enables models with a context window of 1 million tokens to process two hours of 30 FPS video per prompt.

The company will use newly raised capital to purchase additional computing capacity. Additionally, Standard Intelligence plans to develop AI safety guardrails optimized for computer use models. 

Image: Unsplash