Komprise, a reference in analytics-driven unstructured data management, announced Komprise Transparent File Tables, which exposes a structured view of enterprise unstructured data to leading AI, business intelligence and analytics platforms such as Snowflake and Databricks.Data engineers, data scientists and analysts can now query and use unstructured data as an Apache Iceberg table in their familiar environment while avoiding massive costs for large-scale data movement.

Although unstructured data is over 80% of an enterprise’s data footprint, less than 1% of this data is being used in AI, according to IDC. Enterprise unstructured data has remained dark to AI despite its value because it lacks consistent schema; it is of poor quality, and it is large and cumbersome to move. Current ingestion mechanisms are not well-suited for unstructured data because they typically copy all the raw data, which lacks the necessary structure for AI and analytics. Furthermore, ingesting petabytes from multi-vendor storage, especially across NAS and cloud, is extremely complex, expensive, laborious and can take weeks to months to transfer.

Komprise Transparent File Tables leverages the Komprise distributed, scale-out architecture that globally classifies unstructured data to present a tabular high-quality schema.  The table displays Komprise-enriched metadata and a pointer to the data using the patented Komprise Transparent Move Technology (TMT). This enables access to remote data without moving the files, and similar to how Komprise handles transparency for NAS data, Komprise TFT dynamically loads the data just when it is needed. 

Example Use Cases: 

• Analytics dashboards combining structured and unstructured data: A data analyst at a pharmaceutical company can create dashboards in Snowflake or Databricks for their drug research projects by querying a Komprise Transparent File Table for project files generated by each instrument and lab. The analyst can then join the data with financial tables from their ERP systems and instrument information from Benchling, thus combining structured and unstructured data from different sources in a single interface
• AI pipelines combining structured and unstructured data: An AI agent in media and entertainment helping with narrative alignment can use structured project data to identify relevant media archives and join this with Komprise Transparent File Tables to narrow down which scripts to ingest for summarization

How Komprise Transparent File Tables Work: 

• Komprise indexes enterprise data across datacenters and hybrid cloud storage into a Global Metadatabase
• IT users can add rich context to files with content, header and sensitive data scanning and metadata tagging using Komprise AI Preparation and Process Automation (KAPPA) data servicesand Komprise Smart Data Workflows
• IT users make the Global Metadatabase or subsets available in data lakehouses by exporting Komprise Transparent File Tables
• Enterprise data experts can then create queries in Apache Iceberg using their preferred BI and analytics tools without knowledge of or access to Komprise
• Komprise provides data governance based on user access permissions
• If the full files are required for AI, TMT leverages Komprise Intelligent AI Ingest and moves just the needed files at 2X the speed of standard data transfer tools

“The reason 99% of enterprise unstructured data has been dark to AI and analytics is because discovering and generating its schema and moving it is inherently complex and costly,” said Kumar K. Goswami, CEO and co-founder, Komprise. “Komprise brings to light the huge petabytes of enterprise unstructured data in a form that data teams can access easily and transparently for analytics.  Komprise Transparent File Tables opens a whole new world to AI.”

Availability
Komprise Transparent File Tables is now available for early access.