Large language models have dominated headlines, but what about video AI?

TwelveLabs specializes in multimodal AI models that can “watch” and analyze video content. Founded in 2020, the company has built a strong customer base around its video intelligence capabilities.

“Over 80% of the world’s data is now in video, yet it’s the least understood,” said Danny Nicolopoulos (pictured), head of strategic partnerships at TwelveLabs. “Naturally, the companies and industries who monetize on video were the first to raise their hand. And that’s primarily what we call TMEGS, or telco, media, entertainment, games and sports. We now engage with most of the large companies in that industry.”

Nicolopoulos spoke with theCUBE’s Christophe Bertrand for the AWS Marketplace Series, during an exclusive interview on theCUBE, SiliconANGLE Media’s livestreaming studio. They discussed what video AI brings to the table for the entertainment and sports sectors, as well as how TwelveLabs has expanded its reach through AWS Marketplace. (* Disclosure below.)

Video as the new AI frontier

TwelveLabs provides two AI models. The Marengo model is used to search videos, much as a human might watch a video or recall a memory, while Pegasus is a video language model that can analyze video content for further insights. Together, the models form what Nicolopoulos describes as a “powerful librarian.”

“An example I like to give is you’re with a friend and you say, ‘Hey, do you remember that play in the last game where the quarterback threw the touchdown?’” he said. “You’d immediately replay that back in your brain and see it happening. That’s how our models work. That allows us to really, really accurately find moments with semantic language that are very deep, deep in video that otherwise you wouldn’t be able to search for.”

In addition to Marengo and Pegasus, TwelveLabs recently released Rodeo, an AI copilot that helps video editors search and assemble footage with natural language prompts. The suite of capabilities has proven useful to customers such as a Canadian sports company, which uses TwelveLabs’ models to create short highlight reels for social media.

“Rodeo came at the request of customers,” Nicolopoulos explained. “[They] were looking at using the models and building with them independent of each other but specifically wanted it for a more pointed purpose, which was, ‘How do I actually accelerate my editing process to include faster search, analyzing, but also just like a head start on what is otherwise a pretty arduous process?’”

By joining AWS Marketplace, TwelveLabs has reached a wider customer base and discovered further applications for its models beyond the typical entertainment and media sectors. Often, customers already have the data they want analyzed on AWS, but if not, TwelveLabs has a number of integration partners that are part of the AWS ecosystem.

“We’re finding even more use cases,” Nicolopoulos said. “Things like a large auto manufacturer who built an automated mechanic who looks at cars when they come into the shop and provides initial analysis. Law firms trying to understand copious amounts of discovery or investigative evidence. Insurance companies are looking to make a heavily paper-based process into a video-based process that requires less human intervention.”

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s coverage of the AWS Marketplace Series:

(* Disclosure: TheCUBE is a paid media partner for the AWS Marketplace Series. Neither AWS, the sponsor of theCUBE’s event coverage, nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE