Snowflake manager explains the 'Spider-Man' theory of AI agent data access
With access to great data comes great responsibility
Snowflake is betting that the biggest bottleneck to building more and better AI agents isn't the models themselves but whether the data those agents depend on is clean, accessible, and governed, Snowflake’s director of product management James Rowland-Jones told The Register.
He said that the data and AI platform is doubling down on open standards to solve that problem.
Fresh off the Apache Iceberg Summit this week, Rowland-Jones said that Snowflake is working toward "a complete interoperable stack" built around the Apache Iceberg open table format.
“You have essentially data-powered AI platforms and AI powered data platforms,” he said. “But in order for this to work in an AI era, you need to be able to have a set of data that you can get to very easily and accessibly. And that's where the interoperability story really begins because more and more you need to have a single copy of the data.”
Reducing token costs and improving AI agent performance depends on providing agents with a clear, coherent set of context, which he said is only possible when data is available through a unified governance layer.
But that expanded data access introduces new responsibilities, which he called the “Spider-Man story.”
"If I give you direct access to data, you need to be able to act on that data responsibly as well,” Rowland-Jones siad.
He pointed to the Iceberg REST catalog specification and its use of secure vended credentials as the foundation for what he described as technology-neutral, standards-based data access.
“So by having your foundation of your data on an interoperable format and standard like Apache iceberg, and you're using standards like Iceberg REST, and you're using Apache Polaris based kind of governance layers for kind of getting access to that data, what you're doing is then enabling customers to then attach other engines and get multiple, we call multi reader, multi writer, access to that data right areas and directly, irrespective of whether they come through a Snowflake compute engine or not,” he said.
Snowflake's vision, Rowland-Jones said, is to enable access to data stored in cloud object storage, like Amazon S3, regardless of whether the compute engine accessing it is Snowflake's own or a third party's, such as Apache Spark.
"Interoperability without compromise," Rowland-Jones said, describing the goal as allowing customers to use Snowflake's governance capabilities while also supporting other engines directly accessing the same underlying data.
The roadmap includes general availability of Iceberg v3 support, interoperable reads and writes for any engine through Snowflake Horizon Catalog, and a Snowflake-managed storage capability for Iceberg tables.
"We are very passionate about making sure that we contribute to the Iceberg community as well as benefit from it," Rowland-Jones said. "We believe that open source is a two-way street — you can't just consume from it."
He said Snowflake is currently in public preview on Iceberg v3 and what Rowland-Jones called "arguably the broadest coverage of the Iceberg v3 specification" among vendors.
“We have, I would say, very, very strong interest, not just from Snowflake customers, but from the ecosystem on seeing implementations of that,” he told The Register. “And a good example of that would be even across other vendors who are now able to connect to Snowflake and consume Iceberg v3 already. And so we're working very closely with our customers and the community to make all of that a reality.” ®