DuckDB 1.5.3 Iceberg updates, PostgreSQL TDE extension & AI index tuning

Today's Highlights

Today's highlights include DuckDB's enhanced Iceberg integration with new DML and schema evolution features, alongside a deep dive into PostgreSQL's new open-source Transparent Data Encryption. Additionally, we explore AI-driven strategies for automating PostgreSQL index tuning, offering practical performance improvements.

New DuckDB-Iceberg Features in v1.5.3 (DuckDB Blog)

Source: https://duckdb.org/2026/05/29/new-iceberg-features.html

The latest DuckDB v1.5.3 release introduces significant enhancements for working with Apache Iceberg tables, a critical component in modern data lake architectures. Key additions include full MERGE INTO support, allowing users to efficiently update, insert, and delete rows in Iceberg tables based on a source query. This release also brings ALTER TABLE commands for schema evolution, enabling operations like adding, renaming, or dropping columns, crucial for adapting to changing data requirements.
Furthermore, DuckDB now supports partition transforms within Iceberg, providing more control over data organization and query optimization. Compatibility has been extended to Iceberg V3, ensuring support for the latest table format specifications, and improved handling for Iceberg REST Catalogs streamlines metadata management. These features position DuckDB as an even more powerful embedded analytical database for processing large-scale, evolving datasets directly in a data lake environment, making complex ETL/ELT operations more accessible and performant.

Comment: The MERGE INTO and ALTER TABLE additions are game-changers for using DuckDB in production data pipelines with Iceberg, enabling robust upserts and schema changes.

Open-Source TDE for PostgreSQL: What pg_tde Is, and Whether You Need It (Planet PostgreSQL)

Source: https://postgr.es/p/9kM

This article introduces pg_tde, PostgreSQL's new open-source Transparent Data Encryption (TDE) option, a long-awaited feature for enhanced data security at rest. TDE encrypts database files directly, making data unreadable to unauthorized access at the filesystem level without requiring application-level changes. The post delves into what pg_tde provides, including its architecture, key management capabilities, and how it integrates with existing PostgreSQL deployments.
It explains the benefits of TDE for compliance requirements, particularly in regulated industries, and discusses the performance implications and operational considerations of implementing such a security measure. The author also helps readers evaluate whether pg_tde is suitable for their specific use cases, outlining scenarios where it offers critical protection versus situations where other encryption methods might suffice. This is a crucial development for PostgreSQL users looking to strengthen their data security posture without relying on proprietary solutions.

Comment: pg_tde addresses a significant security gap in open-source PostgreSQL, providing a practical, filesystem-level encryption solution that's critical for many compliance-driven environments.

Automating PostgreSQL Index Tuning Using AI (Planet PostgreSQL)

Source: https://postgr.es/p/9kF

This piece explores the emerging field of automating PostgreSQL index tuning through the application of Artificial Intelligence. Traditional index tuning is often a manual, iterative, and expertise-intensive process, involving careful analysis of query plans, workload patterns, and data distribution. The article discusses how AI and machine learning techniques can analyze these factors automatically to suggest optimal index configurations or even dynamically create and drop indexes.
It delves into the methodologies behind such AI-powered systems, potentially covering aspects like reinforcement learning, cost-based optimization with learned models, and predictive analytics to anticipate query performance. The benefits include significant performance gains, reduced operational overhead for DBAs, and improved resource utilization. This approach promises to democratize advanced database optimization, making sophisticated tuning accessible to a broader range of users and potentially leading to more responsive and efficient PostgreSQL deployments.

Comment: Leveraging AI for index tuning is a promising step towards truly self-optimizing databases, potentially saving countless hours for DBAs and improving query performance proactively.